From 8c3313d02e3053a43cb0c4cfba5b5b78376dfa20 Mon Sep 17 00:00:00 2001 From: Jesper Jensen Date: Mon, 22 Dec 2025 22:08:23 +0100 Subject: Add RFCs --- docs/rfc2234.txt | 787 +++++ docs/rfc2616.txt | 9859 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ docs/rfc3986.txt | 3419 +++++++++++++++++++ docs/rfc3987.txt | 2579 ++++++++++++++ docs/rfc5987.txt | 563 ++++ docs/rfc5988.txt | 1291 +++++++ docs/rfc8288.txt | 1347 ++++++++ 7 files changed, 19845 insertions(+) create mode 100644 docs/rfc2234.txt create mode 100644 docs/rfc2616.txt create mode 100644 docs/rfc3986.txt create mode 100644 docs/rfc3987.txt create mode 100644 docs/rfc5987.txt create mode 100644 docs/rfc5988.txt create mode 100644 docs/rfc8288.txt diff --git a/docs/rfc2234.txt b/docs/rfc2234.txt new file mode 100644 index 0000000..edea302 --- /dev/null +++ b/docs/rfc2234.txt @@ -0,0 +1,787 @@ + + + + + + +Network Working Group D. Crocker, Ed. +Request for Comments: 2234 Internet Mail Consortium +Category: Standards Track P. Overell + Demon Internet Ltd. + November 1997 + + + Augmented BNF for Syntax Specifications: ABNF + + +Status of this Memo + + This document specifies an Internet standards track protocol for the + Internet community, and requests discussion and suggestions for + improvements. Please refer to the current edition of the "Internet + Official Protocol Standards" (STD 1) for the standardization state + and status of this protocol. Distribution of this memo is unlimited. + +Copyright Notice + + Copyright (C) The Internet Society (1997). All Rights Reserved. + +TABLE OF CONTENTS + + 1. INTRODUCTION .................................................. 2 + + 2. RULE DEFINITION ............................................... 2 + 2.1 RULE NAMING .................................................. 2 + 2.2 RULE FORM .................................................... 3 + 2.3 TERMINAL VALUES .............................................. 3 + 2.4 EXTERNAL ENCODINGS ........................................... 5 + + 3. OPERATORS ..................................................... 5 + 3.1 CONCATENATION RULE1 RULE2 ............................. 5 + 3.2 ALTERNATIVES RULE1 / RULE2 ................................... 6 + 3.3 INCREMENTAL ALTERNATIVES RULE1 =/ RULE2 .................... 6 + 3.4 VALUE RANGE ALTERNATIVES %C##-## ........................... 7 + 3.5 SEQUENCE GROUP (RULE1 RULE2) ................................. 7 + 3.6 VARIABLE REPETITION *RULE .................................... 8 + 3.7 SPECIFIC REPETITION NRULE .................................... 8 + 3.8 OPTIONAL SEQUENCE [RULE] ..................................... 8 + 3.9 ; COMMENT .................................................... 8 + 3.10 OPERATOR PRECEDENCE ......................................... 9 + + 4. ABNF DEFINITION OF ABNF ....................................... 9 + + 5. SECURITY CONSIDERATIONS ....................................... 10 + + + + +Crocker & Overell Standards Track [Page 1] + +RFC 2234 ABNF for Syntax Specifications November 1997 + + + 6. APPENDIX A - CORE ............................................. 11 + 6.1 CORE RULES ................................................... 11 + 6.2 COMMON ENCODING .............................................. 12 + + 7. ACKNOWLEDGMENTS ............................................... 12 + + 8. REFERENCES .................................................... 13 + + 9. CONTACT ....................................................... 13 + + 10. FULL COPYRIGHT STATEMENT ..................................... 14 + +1. INTRODUCTION + + Internet technical specifications often need to define a format + syntax and are free to employ whatever notation their authors deem + useful. Over the years, a modified version of Backus-Naur Form + (BNF), called Augmented BNF (ABNF), has been popular among many + Internet specifications. It balances compactness and simplicity, + with reasonable representational power. In the early days of the + Arpanet, each specification contained its own definition of ABNF. + This included the email specifications, RFC733 and then RFC822 which + have come to be the common citations for defining ABNF. The current + document separates out that definition, to permit selective + reference. Predictably, it also provides some modifications and + enhancements. + + The differences between standard BNF and ABNF involve naming rules, + repetition, alternatives, order-independence, and value ranges. + Appendix A (Core) supplies rule definitions and encoding for a core + lexical analyzer of the type common to several Internet + specifications. It is provided as a convenience and is otherwise + separate from the meta language defined in the body of this document, + and separate from its formal status. + +2. RULE DEFINITION + +2.1 Rule Naming + + The name of a rule is simply the name itself; that is, a sequence of + characters, beginning with an alphabetic character, and followed by + a combination of alphabetics, digits and hyphens (dashes). + + NOTE: Rule names are case-insensitive + + The names , , and all refer + to the same rule. + + + + +Crocker & Overell Standards Track [Page 2] + +RFC 2234 ABNF for Syntax Specifications November 1997 + + + Unlike original BNF, angle brackets ("<", ">") are not required. + However, angle brackets may be used around a rule name whenever their + presence will facilitate discerning the use of a rule name. This is + typically restricted to rule name references in free-form prose, or + to distinguish partial rules that combine into a string not separated + by white space, such as shown in the discussion about repetition, + below. + +2.2 Rule Form + + A rule is defined by the following sequence: + + name = elements crlf + + where is the name of the rule, is one or more rule + names or terminal specifications and is the end-of- line + indicator, carriage return followed by line feed. The equal sign + separates the name from the definition of the rule. The elements + form a sequence of one or more rule names and/or value definitions, + combined according to the various operators, defined in this + document, such as alternative and repetition. + + For visual ease, rule definitions are left aligned. When a rule + requires multiple lines, the continuation lines are indented. The + left alignment and indentation are relative to the first lines of the + ABNF rules and need not match the left margin of the document. + +2.3 Terminal Values + + Rules resolve into a string of terminal values, sometimes called + characters. In ABNF a character is merely a non-negative integer. + In certain contexts a specific mapping (encoding) of values into a + character set (such as ASCII) will be specified. + + Terminals are specified by one or more numeric characters with the + base interpretation of those characters indicated explicitly. The + following bases are currently defined: + + b = binary + + d = decimal + + x = hexadecimal + + + + + + + + +Crocker & Overell Standards Track [Page 3] + +RFC 2234 ABNF for Syntax Specifications November 1997 + + + Hence: + + CR = %d13 + + CR = %x0D + + respectively specify the decimal and hexadecimal representation of + [US-ASCII] for carriage return. + + A concatenated string of such values is specified compactly, using a + period (".") to indicate separation of characters within that value. + Hence: + + CRLF = %d13.10 + + ABNF permits specifying literal text string directly, enclosed in + quotation-marks. Hence: + + command = "command string" + + Literal text strings are interpreted as a concatenated set of + printable characters. + + NOTE: ABNF strings are case-insensitive and + the character set for these strings is us-ascii. + + Hence: + + rulename = "abc" + + and: + + rulename = "aBc" + + will match "abc", "Abc", "aBc", "abC", "ABc", "aBC", "AbC" and "ABC". + + To specify a rule which IS case SENSITIVE, + specify the characters individually. + + For example: + + rulename = %d97 %d98 %d99 + + or + + rulename = %d97.98.99 + + + + + +Crocker & Overell Standards Track [Page 4] + +RFC 2234 ABNF for Syntax Specifications November 1997 + + + will match only the string which comprises only lowercased + characters, abc. + +2.4 External Encodings + + External representations of terminal value characters will vary + according to constraints in the storage or transmission environment. + Hence, the same ABNF-based grammar may have multiple external + encodings, such as one for a 7-bit US-ASCII environment, another for + a binary octet environment and still a different one when 16-bit + Unicode is used. Encoding details are beyond the scope of ABNF, + although Appendix A (Core) provides definitions for a 7-bit US-ASCII + environment as has been common to much of the Internet. + + By separating external encoding from the syntax, it is intended that + alternate encoding environments can be used for the same syntax. + +3. OPERATORS + +3.1 Concatenation Rule1 Rule2 + + A rule can define a simple, ordered string of values -- i.e., a + concatenation of contiguous characters -- by listing a sequence of + rule names. For example: + + foo = %x61 ; a + + bar = %x62 ; b + + mumble = foo bar foo + + So that the rule matches the lowercase string "aba". + + LINEAR WHITE SPACE: Concatenation is at the core of the ABNF + parsing model. A string of contiguous characters (values) is + parsed according to the rules defined in ABNF. For Internet + specifications, there is some history of permitting linear white + space (space and horizontal tab) to be freelyPand + implicitlyPinterspersed around major constructs, such as + delimiting special characters or atomic strings. + + NOTE: This specification for ABNF does not + provide for implicit specification of linear white + space. + + Any grammar which wishes to permit linear white space around + delimiters or string segments must specify it explicitly. It is + often useful to provide for such white space in "core" rules that are + + + +Crocker & Overell Standards Track [Page 5] + +RFC 2234 ABNF for Syntax Specifications November 1997 + + + then used variously among higher-level rules. The "core" rules might + be formed into a lexical analyzer or simply be part of the main + ruleset. + +3.2 Alternatives Rule1 / Rule2 + + Elements separated by forward slash ("/") are alternatives. + Therefore, + + foo / bar + + will accept or . + + NOTE: A quoted string containing alphabetic + characters is special form for specifying alternative + characters and is interpreted as a non-terminal + representing the set of combinatorial strings with the + contained characters, in the specified order but with + any mixture of upper and lower case.. + +3.3 Incremental Alternatives Rule1 =/ Rule2 + + It is sometimes convenient to specify a list of alternatives in + fragments. That is, an initial rule may match one or more + alternatives, with later rule definitions adding to the set of + alternatives. This is particularly useful for otherwise- independent + specifications which derive from the same parent rule set, such as + often occurs with parameter lists. ABNF permits this incremental + definition through the construct: + + oldrule =/ additional-alternatives + + So that the rule set + + ruleset = alt1 / alt2 + + ruleset =/ alt3 + + ruleset =/ alt4 / alt5 + + is the same as specifying + + ruleset = alt1 / alt2 / alt3 / alt4 / alt5 + + + + + + + + +Crocker & Overell Standards Track [Page 6] + +RFC 2234 ABNF for Syntax Specifications November 1997 + + +3.4 Value Range Alternatives %c##-## + + A range of alternative numeric values can be specified compactly, + using dash ("-") to indicate the range of alternative values. Hence: + + DIGIT = %x30-39 + + is equivalent to: + + DIGIT = "0" / "1" / "2" / "3" / "4" / "5" / "6" / + + "7" / "8" / "9" + + Concatenated numeric values and numeric value ranges can not be + specified in the same string. A numeric value may use the dotted + notation for concatenation or it may use the dash notation to specify + one value range. Hence, to specify one printable character, between + end of line sequences, the specification could be: + + char-line = %x0D.0A %x20-7E %x0D.0A + +3.5 Sequence Group (Rule1 Rule2) + + Elements enclosed in parentheses are treated as a single element, + whose contents are STRICTLY ORDERED. Thus, + + elem (foo / bar) blat + + which matches (elem foo blat) or (elem bar blat). + + elem foo / bar blat + + matches (elem foo) or (bar blat). + + NOTE: It is strongly advised to use grouping + notation, rather than to rely on proper reading of + "bare" alternations, when alternatives consist of + multiple rule names or literals. + + Hence it is recommended that instead of the above form, the form: + + (elem foo) / (bar blat) + + be used. It will avoid misinterpretation by casual readers. + + The sequence group notation is also used within free text to set off + an element sequence from the prose. + + + + +Crocker & Overell Standards Track [Page 7] + +RFC 2234 ABNF for Syntax Specifications November 1997 + + +3.6 Variable Repetition *Rule + + The operator "*" preceding an element indicates repetition. The full + form is: + + *element + + where and are optional decimal values, indicating at least + and at most occurrences of element. + + Default values are 0 and infinity so that * allows any + number, including zero; 1* requires at least one; + 3*3 allows exactly 3 and 1*2 allows one or two. + +3.7 Specific Repetition nRule + + A rule of the form: + + element + + is equivalent to + + *element + + That is, exactly occurrences of . Thus 2DIGIT is a + 2-digit number, and 3ALPHA is a string of three alphabetic + characters. + +3.8 Optional Sequence [RULE] + + Square brackets enclose an optional element sequence: + + [foo bar] + + is equivalent to + + *1(foo bar). + +3.9 ; Comment + + A semi-colon starts a comment that continues to the end of line. + This is a simple way of including useful notes in parallel with the + specifications. + + + + + + + + +Crocker & Overell Standards Track [Page 8] + +RFC 2234 ABNF for Syntax Specifications November 1997 + + +3.10 Operator Precedence + + The various mechanisms described above have the following precedence, + from highest (binding tightest) at the top, to lowest and loosest at + the bottom: + + Strings, Names formation + Comment + Value range + Repetition + Grouping, Optional + Concatenation + Alternative + + Use of the alternative operator, freely mixed with concatenations can + be confusing. + + Again, it is recommended that the grouping operator be used to + make explicit concatenation groups. + +4. ABNF DEFINITION OF ABNF + + This syntax uses the rules provided in Appendix A (Core). + + rulelist = 1*( rule / (*c-wsp c-nl) ) + + rule = rulename defined-as elements c-nl + ; continues if next line starts + ; with white space + + rulename = ALPHA *(ALPHA / DIGIT / "-") + + defined-as = *c-wsp ("=" / "=/") *c-wsp + ; basic rules definition and + ; incremental alternatives + + elements = alternation *c-wsp + + c-wsp = WSP / (c-nl WSP) + + c-nl = comment / CRLF + ; comment or newline + + comment = ";" *(WSP / VCHAR) CRLF + + alternation = concatenation + *(*c-wsp "/" *c-wsp concatenation) + + + + +Crocker & Overell Standards Track [Page 9] + +RFC 2234 ABNF for Syntax Specifications November 1997 + + + concatenation = repetition *(1*c-wsp repetition) + + repetition = [repeat] element + + repeat = 1*DIGIT / (*DIGIT "*" *DIGIT) + + element = rulename / group / option / + char-val / num-val / prose-val + + group = "(" *c-wsp alternation *c-wsp ")" + + option = "[" *c-wsp alternation *c-wsp "]" + + char-val = DQUOTE *(%x20-21 / %x23-7E) DQUOTE + ; quoted string of SP and VCHAR + without DQUOTE + + num-val = "%" (bin-val / dec-val / hex-val) + + bin-val = "b" 1*BIT + [ 1*("." 1*BIT) / ("-" 1*BIT) ] + ; series of concatenated bit values + ; or single ONEOF range + + dec-val = "d" 1*DIGIT + [ 1*("." 1*DIGIT) / ("-" 1*DIGIT) ] + + hex-val = "x" 1*HEXDIG + [ 1*("." 1*HEXDIG) / ("-" 1*HEXDIG) ] + + prose-val = "<" *(%x20-3D / %x3F-7E) ">" + ; bracketed string of SP and VCHAR + without angles + ; prose description, to be used as + last resort + + +5. SECURITY CONSIDERATIONS + + Security is truly believed to be irrelevant to this document. + + + + + + + + + + + +Crocker & Overell Standards Track [Page 10] + +RFC 2234 ABNF for Syntax Specifications November 1997 + + +6. APPENDIX A - CORE + + This Appendix is provided as a convenient core for specific grammars. + The definitions may be used as a core set of rules. + +6.1 Core Rules + + Certain basic rules are in uppercase, such as SP, HTAB, CRLF, + DIGIT, ALPHA, etc. + + ALPHA = %x41-5A / %x61-7A ; A-Z / a-z + + BIT = "0" / "1" + + CHAR = %x01-7F + ; any 7-bit US-ASCII character, + excluding NUL + + CR = %x0D + ; carriage return + + CRLF = CR LF + ; Internet standard newline + + CTL = %x00-1F / %x7F + ; controls + + DIGIT = %x30-39 + ; 0-9 + + DQUOTE = %x22 + ; " (Double Quote) + + HEXDIG = DIGIT / "A" / "B" / "C" / "D" / "E" / "F" + + HTAB = %x09 + ; horizontal tab + + LF = %x0A + ; linefeed + + LWSP = *(WSP / CRLF WSP) + ; linear white space (past newline) + + OCTET = %x00-FF + ; 8 bits of data + + SP = %x20 + + + +Crocker & Overell Standards Track [Page 11] + +RFC 2234 ABNF for Syntax Specifications November 1997 + + + ; space + + VCHAR = %x21-7E + ; visible (printing) characters + + WSP = SP / HTAB + ; white space + +6.2 Common Encoding + + Externally, data are represented as "network virtual ASCII", namely + 7-bit US-ASCII in an 8-bit field, with the high (8th) bit set to + zero. A string of values is in "network byte order" with the + higher-valued bytes represented on the left-hand side and being sent + over the network first. + +7. ACKNOWLEDGMENTS + + The syntax for ABNF was originally specified in RFC 733. Ken L. + Harrenstien, of SRI International, was responsible for re-coding the + BNF into an augmented BNF that makes the representation smaller and + easier to understand. + + This recent project began as a simple effort to cull out the portion + of RFC 822 which has been repeatedly cited by non-email specification + writers, namely the description of augmented BNF. Rather than simply + and blindly converting the existing text into a separate document, + the working group chose to give careful consideration to the + deficiencies, as well as benefits, of the existing specification and + related specifications available over the last 15 years and therefore + to pursue enhancement. This turned the project into something rather + more ambitious than first intended. Interestingly the result is not + massively different from that original, although decisions such as + removing the list notation came as a surprise. + + The current round of specification was part of the DRUMS working + group, with significant contributions from Jerome Abela , Harald + Alvestrand, Robert Elz, Roger Fajman, Aviva Garrett, Tom Harsch, Dan + Kohn, Bill McQuillan, Keith Moore, Chris Newman , Pete Resnick and + Henning Schulzrinne. + + + + + + + + + + + +Crocker & Overell Standards Track [Page 12] + +RFC 2234 ABNF for Syntax Specifications November 1997 + + +8. REFERENCES + + [US-ASCII] Coded Character Set--7-Bit American Standard Code for + Information Interchange, ANSI X3.4-1986. + + [RFC733] Crocker, D., Vittal, J., Pogran, K., and D. Henderson, + "Standard for the Format of ARPA Network Text Message," RFC 733, + November 1977. + + [RFC822] Crocker, D., "Standard for the Format of ARPA Internet Text + Messages", STD 11, RFC 822, August 1982. + +9. CONTACT + + David H. Crocker Paul Overell + + Internet Mail Consortium Demon Internet Ltd + 675 Spruce Dr. Dorking Business Park + Sunnyvale, CA 94086 USA Dorking + Surrey, RH4 1HN + UK + + Phone: +1 408 246 8253 + Fax: +1 408 249 6205 + EMail: dcrocker@imc.org paulo@turnpike.com + + + + + + + + + + + + + + + + + + + + + + + + + + +Crocker & Overell Standards Track [Page 13] + +RFC 2234 ABNF for Syntax Specifications November 1997 + + +10. Full Copyright Statement + + Copyright (C) The Internet Society (1997). All Rights Reserved. + + This document and translations of it may be copied and furnished to + others, and derivative works that comment on or otherwise explain it + or assist in its implementation may be prepared, copied, published + and distributed, in whole or in part, without restriction of any + kind, provided that the above copyright notice and this paragraph are + included on all such copies and derivative works. However, this + document itself may not be modified in any way, such as by removing + the copyright notice or references to the Internet Society or other + Internet organizations, except as needed for the purpose of + developing Internet standards in which case the procedures for + copyrights defined in the Internet Standards process must be + followed, or as required to translate it into languages other than + English. + + The limited permissions granted above are perpetual and will not be + revoked by the Internet Society or its successors or assigns. + + This document and the information contained herein is provided on an + "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING + TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING + BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION + HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF + MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. + + + + + + + + + + + + + + + + + + + + + + + + +Crocker & Overell Standards Track [Page 14] + diff --git a/docs/rfc2616.txt b/docs/rfc2616.txt new file mode 100644 index 0000000..45d7d08 --- /dev/null +++ b/docs/rfc2616.txt @@ -0,0 +1,9859 @@ + + + + + + +Network Working Group R. Fielding +Request for Comments: 2616 UC Irvine +Obsoletes: 2068 J. Gettys +Category: Standards Track Compaq/W3C + J. Mogul + Compaq + H. Frystyk + W3C/MIT + L. Masinter + Xerox + P. Leach + Microsoft + T. Berners-Lee + W3C/MIT + June 1999 + + + Hypertext Transfer Protocol -- HTTP/1.1 + +Status of this Memo + + This document specifies an Internet standards track protocol for the + Internet community, and requests discussion and suggestions for + improvements. Please refer to the current edition of the "Internet + Official Protocol Standards" (STD 1) for the standardization state + and status of this protocol. Distribution of this memo is unlimited. + +Copyright Notice + + Copyright (C) The Internet Society (1999). All Rights Reserved. + +Abstract + + The Hypertext Transfer Protocol (HTTP) is an application-level + protocol for distributed, collaborative, hypermedia information + systems. It is a generic, stateless, protocol which can be used for + many tasks beyond its use for hypertext, such as name servers and + distributed object management systems, through extension of its + request methods, error codes and headers [47]. A feature of HTTP is + the typing and negotiation of data representation, allowing systems + to be built independently of the data being transferred. + + HTTP has been in use by the World-Wide Web global information + initiative since 1990. This specification defines the protocol + referred to as "HTTP/1.1", and is an update to RFC 2068 [33]. + + + + + + +Fielding, et al. Standards Track [Page 1] + +RFC 2616 HTTP/1.1 June 1999 + + +Table of Contents + + 1 Introduction ...................................................7 + 1.1 Purpose......................................................7 + 1.2 Requirements .................................................8 + 1.3 Terminology ..................................................8 + 1.4 Overall Operation ...........................................12 + 2 Notational Conventions and Generic Grammar ....................14 + 2.1 Augmented BNF ...............................................14 + 2.2 Basic Rules .................................................15 + 3 Protocol Parameters ...........................................17 + 3.1 HTTP Version ................................................17 + 3.2 Uniform Resource Identifiers ................................18 + 3.2.1 General Syntax ...........................................19 + 3.2.2 http URL .................................................19 + 3.2.3 URI Comparison ...........................................20 + 3.3 Date/Time Formats ...........................................20 + 3.3.1 Full Date ................................................20 + 3.3.2 Delta Seconds ............................................21 + 3.4 Character Sets ..............................................21 + 3.4.1 Missing Charset ..........................................22 + 3.5 Content Codings .............................................23 + 3.6 Transfer Codings ............................................24 + 3.6.1 Chunked Transfer Coding ..................................25 + 3.7 Media Types .................................................26 + 3.7.1 Canonicalization and Text Defaults .......................27 + 3.7.2 Multipart Types ..........................................27 + 3.8 Product Tokens ..............................................28 + 3.9 Quality Values ..............................................29 + 3.10 Language Tags ...............................................29 + 3.11 Entity Tags .................................................30 + 3.12 Range Units .................................................30 + 4 HTTP Message ..................................................31 + 4.1 Message Types ...............................................31 + 4.2 Message Headers .............................................31 + 4.3 Message Body ................................................32 + 4.4 Message Length ..............................................33 + 4.5 General Header Fields .......................................34 + 5 Request .......................................................35 + 5.1 Request-Line ................................................35 + 5.1.1 Method ...................................................36 + 5.1.2 Request-URI ..............................................36 + 5.2 The Resource Identified by a Request ........................38 + 5.3 Request Header Fields .......................................38 + 6 Response ......................................................39 + 6.1 Status-Line .................................................39 + 6.1.1 Status Code and Reason Phrase ............................39 + 6.2 Response Header Fields ......................................41 + + + +Fielding, et al. Standards Track [Page 2] + +RFC 2616 HTTP/1.1 June 1999 + + + 7 Entity ........................................................42 + 7.1 Entity Header Fields ........................................42 + 7.2 Entity Body .................................................43 + 7.2.1 Type .....................................................43 + 7.2.2 Entity Length ............................................43 + 8 Connections ...................................................44 + 8.1 Persistent Connections ......................................44 + 8.1.1 Purpose ..................................................44 + 8.1.2 Overall Operation ........................................45 + 8.1.3 Proxy Servers ............................................46 + 8.1.4 Practical Considerations .................................46 + 8.2 Message Transmission Requirements ...........................47 + 8.2.1 Persistent Connections and Flow Control ..................47 + 8.2.2 Monitoring Connections for Error Status Messages .........48 + 8.2.3 Use of the 100 (Continue) Status .........................48 + 8.2.4 Client Behavior if Server Prematurely Closes Connection ..50 + 9 Method Definitions ............................................51 + 9.1 Safe and Idempotent Methods .................................51 + 9.1.1 Safe Methods .............................................51 + 9.1.2 Idempotent Methods .......................................51 + 9.2 OPTIONS .....................................................52 + 9.3 GET .........................................................53 + 9.4 HEAD ........................................................54 + 9.5 POST ........................................................54 + 9.6 PUT .........................................................55 + 9.7 DELETE ......................................................56 + 9.8 TRACE .......................................................56 + 9.9 CONNECT .....................................................57 + 10 Status Code Definitions ......................................57 + 10.1 Informational 1xx ...........................................57 + 10.1.1 100 Continue .............................................58 + 10.1.2 101 Switching Protocols ..................................58 + 10.2 Successful 2xx ..............................................58 + 10.2.1 200 OK ...................................................58 + 10.2.2 201 Created ..............................................59 + 10.2.3 202 Accepted .............................................59 + 10.2.4 203 Non-Authoritative Information ........................59 + 10.2.5 204 No Content ...........................................60 + 10.2.6 205 Reset Content ........................................60 + 10.2.7 206 Partial Content ......................................60 + 10.3 Redirection 3xx .............................................61 + 10.3.1 300 Multiple Choices .....................................61 + 10.3.2 301 Moved Permanently ....................................62 + 10.3.3 302 Found ................................................62 + 10.3.4 303 See Other ............................................63 + 10.3.5 304 Not Modified .........................................63 + 10.3.6 305 Use Proxy ............................................64 + 10.3.7 306 (Unused) .............................................64 + + + +Fielding, et al. Standards Track [Page 3] + +RFC 2616 HTTP/1.1 June 1999 + + + 10.3.8 307 Temporary Redirect ...................................65 + 10.4 Client Error 4xx ............................................65 + 10.4.1 400 Bad Request .........................................65 + 10.4.2 401 Unauthorized ........................................66 + 10.4.3 402 Payment Required ....................................66 + 10.4.4 403 Forbidden ...........................................66 + 10.4.5 404 Not Found ...........................................66 + 10.4.6 405 Method Not Allowed ..................................66 + 10.4.7 406 Not Acceptable ......................................67 + 10.4.8 407 Proxy Authentication Required .......................67 + 10.4.9 408 Request Timeout .....................................67 + 10.4.10 409 Conflict ............................................67 + 10.4.11 410 Gone ................................................68 + 10.4.12 411 Length Required .....................................68 + 10.4.13 412 Precondition Failed .................................68 + 10.4.14 413 Request Entity Too Large ............................69 + 10.4.15 414 Request-URI Too Long ................................69 + 10.4.16 415 Unsupported Media Type ..............................69 + 10.4.17 416 Requested Range Not Satisfiable .....................69 + 10.4.18 417 Expectation Failed ..................................70 + 10.5 Server Error 5xx ............................................70 + 10.5.1 500 Internal Server Error ................................70 + 10.5.2 501 Not Implemented ......................................70 + 10.5.3 502 Bad Gateway ..........................................70 + 10.5.4 503 Service Unavailable ..................................70 + 10.5.5 504 Gateway Timeout ......................................71 + 10.5.6 505 HTTP Version Not Supported ...........................71 + 11 Access Authentication ........................................71 + 12 Content Negotiation ..........................................71 + 12.1 Server-driven Negotiation ...................................72 + 12.2 Agent-driven Negotiation ....................................73 + 12.3 Transparent Negotiation .....................................74 + 13 Caching in HTTP ..............................................74 + 13.1.1 Cache Correctness ........................................75 + 13.1.2 Warnings .................................................76 + 13.1.3 Cache-control Mechanisms .................................77 + 13.1.4 Explicit User Agent Warnings .............................78 + 13.1.5 Exceptions to the Rules and Warnings .....................78 + 13.1.6 Client-controlled Behavior ...............................79 + 13.2 Expiration Model ............................................79 + 13.2.1 Server-Specified Expiration ..............................79 + 13.2.2 Heuristic Expiration .....................................80 + 13.2.3 Age Calculations .........................................80 + 13.2.4 Expiration Calculations ..................................83 + 13.2.5 Disambiguating Expiration Values .........................84 + 13.2.6 Disambiguating Multiple Responses ........................84 + 13.3 Validation Model ............................................85 + 13.3.1 Last-Modified Dates ......................................86 + + + +Fielding, et al. Standards Track [Page 4] + +RFC 2616 HTTP/1.1 June 1999 + + + 13.3.2 Entity Tag Cache Validators ..............................86 + 13.3.3 Weak and Strong Validators ...............................86 + 13.3.4 Rules for When to Use Entity Tags and Last-Modified Dates.89 + 13.3.5 Non-validating Conditionals ..............................90 + 13.4 Response Cacheability .......................................91 + 13.5 Constructing Responses From Caches ..........................92 + 13.5.1 End-to-end and Hop-by-hop Headers ........................92 + 13.5.2 Non-modifiable Headers ...................................92 + 13.5.3 Combining Headers ........................................94 + 13.5.4 Combining Byte Ranges ....................................95 + 13.6 Caching Negotiated Responses ................................95 + 13.7 Shared and Non-Shared Caches ................................96 + 13.8 Errors or Incomplete Response Cache Behavior ................97 + 13.9 Side Effects of GET and HEAD ................................97 + 13.10 Invalidation After Updates or Deletions ...................97 + 13.11 Write-Through Mandatory ...................................98 + 13.12 Cache Replacement .........................................99 + 13.13 History Lists .............................................99 + 14 Header Field Definitions ....................................100 + 14.1 Accept .....................................................100 + 14.2 Accept-Charset .............................................102 + 14.3 Accept-Encoding ............................................102 + 14.4 Accept-Language ............................................104 + 14.5 Accept-Ranges ..............................................105 + 14.6 Age ........................................................106 + 14.7 Allow ......................................................106 + 14.8 Authorization ..............................................107 + 14.9 Cache-Control ..............................................108 + 14.9.1 What is Cacheable .......................................109 + 14.9.2 What May be Stored by Caches ............................110 + 14.9.3 Modifications of the Basic Expiration Mechanism .........111 + 14.9.4 Cache Revalidation and Reload Controls ..................113 + 14.9.5 No-Transform Directive ..................................115 + 14.9.6 Cache Control Extensions ................................116 + 14.10 Connection ...............................................117 + 14.11 Content-Encoding .........................................118 + 14.12 Content-Language .........................................118 + 14.13 Content-Length ...........................................119 + 14.14 Content-Location .........................................120 + 14.15 Content-MD5 ..............................................121 + 14.16 Content-Range ............................................122 + 14.17 Content-Type .............................................124 + 14.18 Date .....................................................124 + 14.18.1 Clockless Origin Server Operation ......................125 + 14.19 ETag .....................................................126 + 14.20 Expect ...................................................126 + 14.21 Expires ..................................................127 + 14.22 From .....................................................128 + + + +Fielding, et al. Standards Track [Page 5] + +RFC 2616 HTTP/1.1 June 1999 + + + 14.23 Host .....................................................128 + 14.24 If-Match .................................................129 + 14.25 If-Modified-Since ........................................130 + 14.26 If-None-Match ............................................132 + 14.27 If-Range .................................................133 + 14.28 If-Unmodified-Since ......................................134 + 14.29 Last-Modified ............................................134 + 14.30 Location .................................................135 + 14.31 Max-Forwards .............................................136 + 14.32 Pragma ...................................................136 + 14.33 Proxy-Authenticate .......................................137 + 14.34 Proxy-Authorization ......................................137 + 14.35 Range ....................................................138 + 14.35.1 Byte Ranges ...........................................138 + 14.35.2 Range Retrieval Requests ..............................139 + 14.36 Referer ..................................................140 + 14.37 Retry-After ..............................................141 + 14.38 Server ...................................................141 + 14.39 TE .......................................................142 + 14.40 Trailer ..................................................143 + 14.41 Transfer-Encoding..........................................143 + 14.42 Upgrade ..................................................144 + 14.43 User-Agent ...............................................145 + 14.44 Vary .....................................................145 + 14.45 Via ......................................................146 + 14.46 Warning ..................................................148 + 14.47 WWW-Authenticate .........................................150 + 15 Security Considerations .......................................150 + 15.1 Personal Information....................................151 + 15.1.1 Abuse of Server Log Information .........................151 + 15.1.2 Transfer of Sensitive Information .......................151 + 15.1.3 Encoding Sensitive Information in URI's .................152 + 15.1.4 Privacy Issues Connected to Accept Headers ..............152 + 15.2 Attacks Based On File and Path Names .......................153 + 15.3 DNS Spoofing ...............................................154 + 15.4 Location Headers and Spoofing ..............................154 + 15.5 Content-Disposition Issues .................................154 + 15.6 Authentication Credentials and Idle Clients ................155 + 15.7 Proxies and Caching ........................................155 + 15.7.1 Denial of Service Attacks on Proxies....................156 + 16 Acknowledgments .............................................156 + 17 References ..................................................158 + 18 Authors' Addresses ..........................................162 + 19 Appendices ..................................................164 + 19.1 Internet Media Type message/http and application/http ......164 + 19.2 Internet Media Type multipart/byteranges ...................165 + 19.3 Tolerant Applications ......................................166 + 19.4 Differences Between HTTP Entities and RFC 2045 Entities ....167 + + + +Fielding, et al. Standards Track [Page 6] + +RFC 2616 HTTP/1.1 June 1999 + + + 19.4.1 MIME-Version ............................................167 + 19.4.2 Conversion to Canonical Form ............................167 + 19.4.3 Conversion of Date Formats ..............................168 + 19.4.4 Introduction of Content-Encoding ........................168 + 19.4.5 No Content-Transfer-Encoding ............................168 + 19.4.6 Introduction of Transfer-Encoding .......................169 + 19.4.7 MHTML and Line Length Limitations .......................169 + 19.5 Additional Features ........................................169 + 19.5.1 Content-Disposition .....................................170 + 19.6 Compatibility with Previous Versions .......................170 + 19.6.1 Changes from HTTP/1.0 ...................................171 + 19.6.2 Compatibility with HTTP/1.0 Persistent Connections ......172 + 19.6.3 Changes from RFC 2068 ...................................172 + 20 Index .......................................................175 + 21 Full Copyright Statement ....................................176 + +1 Introduction + +1.1 Purpose + + The Hypertext Transfer Protocol (HTTP) is an application-level + protocol for distributed, collaborative, hypermedia information + systems. HTTP has been in use by the World-Wide Web global + information initiative since 1990. The first version of HTTP, + referred to as HTTP/0.9, was a simple protocol for raw data transfer + across the Internet. HTTP/1.0, as defined by RFC 1945 [6], improved + the protocol by allowing messages to be in the format of MIME-like + messages, containing metainformation about the data transferred and + modifiers on the request/response semantics. However, HTTP/1.0 does + not sufficiently take into consideration the effects of hierarchical + proxies, caching, the need for persistent connections, or virtual + hosts. In addition, the proliferation of incompletely-implemented + applications calling themselves "HTTP/1.0" has necessitated a + protocol version change in order for two communicating applications + to determine each other's true capabilities. + + This specification defines the protocol referred to as "HTTP/1.1". + This protocol includes more stringent requirements than HTTP/1.0 in + order to ensure reliable implementation of its features. + + Practical information systems require more functionality than simple + retrieval, including search, front-end update, and annotation. HTTP + allows an open-ended set of methods and headers that indicate the + purpose of a request [47]. It builds on the discipline of reference + provided by the Uniform Resource Identifier (URI) [3], as a location + (URL) [4] or name (URN) [20], for indicating the resource to which a + + + + + +Fielding, et al. Standards Track [Page 7] + +RFC 2616 HTTP/1.1 June 1999 + + + method is to be applied. Messages are passed in a format similar to + that used by Internet mail [9] as defined by the Multipurpose + Internet Mail Extensions (MIME) [7]. + + HTTP is also used as a generic protocol for communication between + user agents and proxies/gateways to other Internet systems, including + those supported by the SMTP [16], NNTP [13], FTP [18], Gopher [2], + and WAIS [10] protocols. In this way, HTTP allows basic hypermedia + access to resources available from diverse applications. + +1.2 Requirements + + The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", + "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this + document are to be interpreted as described in RFC 2119 [34]. + + An implementation is not compliant if it fails to satisfy one or more + of the MUST or REQUIRED level requirements for the protocols it + implements. An implementation that satisfies all the MUST or REQUIRED + level and all the SHOULD level requirements for its protocols is said + to be "unconditionally compliant"; one that satisfies all the MUST + level requirements but not all the SHOULD level requirements for its + protocols is said to be "conditionally compliant." + +1.3 Terminology + + This specification uses a number of terms to refer to the roles + played by participants in, and objects of, the HTTP communication. + + connection + A transport layer virtual circuit established between two programs + for the purpose of communication. + + message + The basic unit of HTTP communication, consisting of a structured + sequence of octets matching the syntax defined in section 4 and + transmitted via the connection. + + request + An HTTP request message, as defined in section 5. + + response + An HTTP response message, as defined in section 6. + + + + + + + + +Fielding, et al. Standards Track [Page 8] + +RFC 2616 HTTP/1.1 June 1999 + + + resource + A network data object or service that can be identified by a URI, + as defined in section 3.2. Resources may be available in multiple + representations (e.g. multiple languages, data formats, size, and + resolutions) or vary in other ways. + + entity + The information transferred as the payload of a request or + response. An entity consists of metainformation in the form of + entity-header fields and content in the form of an entity-body, as + described in section 7. + + representation + An entity included with a response that is subject to content + negotiation, as described in section 12. There may exist multiple + representations associated with a particular response status. + + content negotiation + The mechanism for selecting the appropriate representation when + servicing a request, as described in section 12. The + representation of entities in any response can be negotiated + (including error responses). + + variant + A resource may have one, or more than one, representation(s) + associated with it at any given instant. Each of these + representations is termed a `varriant'. Use of the term `variant' + does not necessarily imply that the resource is subject to content + negotiation. + + client + A program that establishes connections for the purpose of sending + requests. + + user agent + The client which initiates a request. These are often browsers, + editors, spiders (web-traversing robots), or other end user tools. + + server + An application program that accepts connections in order to + service requests by sending back responses. Any given program may + be capable of being both a client and a server; our use of these + terms refers only to the role being performed by the program for a + particular connection, rather than to the program's capabilities + in general. Likewise, any server may act as an origin server, + proxy, gateway, or tunnel, switching behavior based on the nature + of each request. + + + + +Fielding, et al. Standards Track [Page 9] + +RFC 2616 HTTP/1.1 June 1999 + + + origin server + The server on which a given resource resides or is to be created. + + proxy + An intermediary program which acts as both a server and a client + for the purpose of making requests on behalf of other clients. + Requests are serviced internally or by passing them on, with + possible translation, to other servers. A proxy MUST implement + both the client and server requirements of this specification. A + "transparent proxy" is a proxy that does not modify the request or + response beyond what is required for proxy authentication and + identification. A "non-transparent proxy" is a proxy that modifies + the request or response in order to provide some added service to + the user agent, such as group annotation services, media type + transformation, protocol reduction, or anonymity filtering. Except + where either transparent or non-transparent behavior is explicitly + stated, the HTTP proxy requirements apply to both types of + proxies. + + gateway + A server which acts as an intermediary for some other server. + Unlike a proxy, a gateway receives requests as if it were the + origin server for the requested resource; the requesting client + may not be aware that it is communicating with a gateway. + + tunnel + An intermediary program which is acting as a blind relay between + two connections. Once active, a tunnel is not considered a party + to the HTTP communication, though the tunnel may have been + initiated by an HTTP request. The tunnel ceases to exist when both + ends of the relayed connections are closed. + + cache + A program's local store of response messages and the subsystem + that controls its message storage, retrieval, and deletion. A + cache stores cacheable responses in order to reduce the response + time and network bandwidth consumption on future, equivalent + requests. Any client or server may include a cache, though a cache + cannot be used by a server that is acting as a tunnel. + + cacheable + A response is cacheable if a cache is allowed to store a copy of + the response message for use in answering subsequent requests. The + rules for determining the cacheability of HTTP responses are + defined in section 13. Even if a resource is cacheable, there may + be additional constraints on whether a cache can use the cached + copy for a particular request. + + + + +Fielding, et al. Standards Track [Page 10] + +RFC 2616 HTTP/1.1 June 1999 + + + first-hand + A response is first-hand if it comes directly and without + unnecessary delay from the origin server, perhaps via one or more + proxies. A response is also first-hand if its validity has just + been checked directly with the origin server. + + explicit expiration time + The time at which the origin server intends that an entity should + no longer be returned by a cache without further validation. + + heuristic expiration time + An expiration time assigned by a cache when no explicit expiration + time is available. + + age + The age of a response is the time since it was sent by, or + successfully validated with, the origin server. + + freshness lifetime + The length of time between the generation of a response and its + expiration time. + + fresh + A response is fresh if its age has not yet exceeded its freshness + lifetime. + + stale + A response is stale if its age has passed its freshness lifetime. + + semantically transparent + A cache behaves in a "semantically transparent" manner, with + respect to a particular response, when its use affects neither the + requesting client nor the origin server, except to improve + performance. When a cache is semantically transparent, the client + receives exactly the same response (except for hop-by-hop headers) + that it would have received had its request been handled directly + by the origin server. + + validator + A protocol element (e.g., an entity tag or a Last-Modified time) + that is used to find out whether a cache entry is an equivalent + copy of an entity. + + upstream/downstream + Upstream and downstream describe the flow of a message: all + messages flow from upstream to downstream. + + + + + +Fielding, et al. Standards Track [Page 11] + +RFC 2616 HTTP/1.1 June 1999 + + + inbound/outbound + Inbound and outbound refer to the request and response paths for + messages: "inbound" means "traveling toward the origin server", + and "outbound" means "traveling toward the user agent" + +1.4 Overall Operation + + The HTTP protocol is a request/response protocol. A client sends a + request to the server in the form of a request method, URI, and + protocol version, followed by a MIME-like message containing request + modifiers, client information, and possible body content over a + connection with a server. The server responds with a status line, + including the message's protocol version and a success or error code, + followed by a MIME-like message containing server information, entity + metainformation, and possible entity-body content. The relationship + between HTTP and MIME is described in appendix 19.4. + + Most HTTP communication is initiated by a user agent and consists of + a request to be applied to a resource on some origin server. In the + simplest case, this may be accomplished via a single connection (v) + between the user agent (UA) and the origin server (O). + + request chain ------------------------> + UA -------------------v------------------- O + <----------------------- response chain + + A more complicated situation occurs when one or more intermediaries + are present in the request/response chain. There are three common + forms of intermediary: proxy, gateway, and tunnel. A proxy is a + forwarding agent, receiving requests for a URI in its absolute form, + rewriting all or part of the message, and forwarding the reformatted + request toward the server identified by the URI. A gateway is a + receiving agent, acting as a layer above some other server(s) and, if + necessary, translating the requests to the underlying server's + protocol. A tunnel acts as a relay point between two connections + without changing the messages; tunnels are used when the + communication needs to pass through an intermediary (such as a + firewall) even when the intermediary cannot understand the contents + of the messages. + + request chain --------------------------------------> + UA -----v----- A -----v----- B -----v----- C -----v----- O + <------------------------------------- response chain + + The figure above shows three intermediaries (A, B, and C) between the + user agent and origin server. A request or response message that + travels the whole chain will pass through four separate connections. + This distinction is important because some HTTP communication options + + + +Fielding, et al. Standards Track [Page 12] + +RFC 2616 HTTP/1.1 June 1999 + + + may apply only to the connection with the nearest, non-tunnel + neighbor, only to the end-points of the chain, or to all connections + along the chain. Although the diagram is linear, each participant may + be engaged in multiple, simultaneous communications. For example, B + may be receiving requests from many clients other than A, and/or + forwarding requests to servers other than C, at the same time that it + is handling A's request. + + Any party to the communication which is not acting as a tunnel may + employ an internal cache for handling requests. The effect of a cache + is that the request/response chain is shortened if one of the + participants along the chain has a cached response applicable to that + request. The following illustrates the resulting chain if B has a + cached copy of an earlier response from O (via C) for a request which + has not been cached by UA or A. + + request chain ----------> + UA -----v----- A -----v----- B - - - - - - C - - - - - - O + <--------- response chain + + Not all responses are usefully cacheable, and some requests may + contain modifiers which place special requirements on cache behavior. + HTTP requirements for cache behavior and cacheable responses are + defined in section 13. + + In fact, there are a wide variety of architectures and configurations + of caches and proxies currently being experimented with or deployed + across the World Wide Web. These systems include national hierarchies + of proxy caches to save transoceanic bandwidth, systems that + broadcast or multicast cache entries, organizations that distribute + subsets of cached data via CD-ROM, and so on. HTTP systems are used + in corporate intranets over high-bandwidth links, and for access via + PDAs with low-power radio links and intermittent connectivity. The + goal of HTTP/1.1 is to support the wide diversity of configurations + already deployed while introducing protocol constructs that meet the + needs of those who build web applications that require high + reliability and, failing that, at least reliable indications of + failure. + + HTTP communication usually takes place over TCP/IP connections. The + default port is TCP 80 [19], but other ports can be used. This does + not preclude HTTP from being implemented on top of any other protocol + on the Internet, or on other networks. HTTP only presumes a reliable + transport; any protocol that provides such guarantees can be used; + the mapping of the HTTP/1.1 request and response structures onto the + transport data units of the protocol in question is outside the scope + of this specification. + + + + +Fielding, et al. Standards Track [Page 13] + +RFC 2616 HTTP/1.1 June 1999 + + + In HTTP/1.0, most implementations used a new connection for each + request/response exchange. In HTTP/1.1, a connection may be used for + one or more request/response exchanges, although connections may be + closed for a variety of reasons (see section 8.1). + +2 Notational Conventions and Generic Grammar + +2.1 Augmented BNF + + All of the mechanisms specified in this document are described in + both prose and an augmented Backus-Naur Form (BNF) similar to that + used by RFC 822 [9]. Implementors will need to be familiar with the + notation in order to understand this specification. The augmented BNF + includes the following constructs: + + name = definition + The name of a rule is simply the name itself (without any + enclosing "<" and ">") and is separated from its definition by the + equal "=" character. White space is only significant in that + indentation of continuation lines is used to indicate a rule + definition that spans more than one line. Certain basic rules are + in uppercase, such as SP, LWS, HT, CRLF, DIGIT, ALPHA, etc. Angle + brackets are used within definitions whenever their presence will + facilitate discerning the use of rule names. + + "literal" + Quotation marks surround literal text. Unless stated otherwise, + the text is case-insensitive. + + rule1 | rule2 + Elements separated by a bar ("|") are alternatives, e.g., "yes | + no" will accept yes or no. + + (rule1 rule2) + Elements enclosed in parentheses are treated as a single element. + Thus, "(elem (foo | bar) elem)" allows the token sequences "elem + foo elem" and "elem bar elem". + + *rule + The character "*" preceding an element indicates repetition. The + full form is "*element" indicating at least and at most + occurrences of element. Default values are 0 and infinity so + that "*(element)" allows any number, including zero; "1*element" + requires at least one; and "1*2element" allows one or two. + + [rule] + Square brackets enclose optional elements; "[foo bar]" is + equivalent to "*1(foo bar)". + + + +Fielding, et al. Standards Track [Page 14] + +RFC 2616 HTTP/1.1 June 1999 + + + N rule + Specific repetition: "(element)" is equivalent to + "*(element)"; that is, exactly occurrences of (element). + Thus 2DIGIT is a 2-digit number, and 3ALPHA is a string of three + alphabetic characters. + + #rule + A construct "#" is defined, similar to "*", for defining lists of + elements. The full form is "#element" indicating at least + and at most elements, each separated by one or more commas + (",") and OPTIONAL linear white space (LWS). This makes the usual + form of lists very easy; a rule such as + ( *LWS element *( *LWS "," *LWS element )) + can be shown as + 1#element + Wherever this construct is used, null elements are allowed, but do + not contribute to the count of elements present. That is, + "(element), , (element) " is permitted, but counts as only two + elements. Therefore, where at least one element is required, at + least one non-null element MUST be present. Default values are 0 + and infinity so that "#element" allows any number, including zero; + "1#element" requires at least one; and "1#2element" allows one or + two. + + ; comment + A semi-colon, set off some distance to the right of rule text, + starts a comment that continues to the end of line. This is a + simple way of including useful notes in parallel with the + specifications. + + implied *LWS + The grammar described by this specification is word-based. Except + where noted otherwise, linear white space (LWS) can be included + between any two adjacent words (token or quoted-string), and + between adjacent words and separators, without changing the + interpretation of a field. At least one delimiter (LWS and/or + + separators) MUST exist between any two tokens (for the definition + of "token" below), since they would otherwise be interpreted as a + single token. + +2.2 Basic Rules + + The following rules are used throughout this specification to + describe basic parsing constructs. The US-ASCII coded character set + is defined by ANSI X3.4-1986 [21]. + + + + + +Fielding, et al. Standards Track [Page 15] + +RFC 2616 HTTP/1.1 June 1999 + + + OCTET = + CHAR = + UPALPHA = + LOALPHA = + ALPHA = UPALPHA | LOALPHA + DIGIT = + CTL = + CR = + LF = + SP = + HT = + <"> = + + HTTP/1.1 defines the sequence CR LF as the end-of-line marker for all + protocol elements except the entity-body (see appendix 19.3 for + tolerant applications). The end-of-line marker within an entity-body + is defined by its associated media type, as described in section 3.7. + + CRLF = CR LF + + HTTP/1.1 header field values can be folded onto multiple lines if the + continuation line begins with a space or horizontal tab. All linear + white space, including folding, has the same semantics as SP. A + recipient MAY replace any linear white space with a single SP before + interpreting the field value or forwarding the message downstream. + + LWS = [CRLF] 1*( SP | HT ) + + The TEXT rule is only used for descriptive field contents and values + that are not intended to be interpreted by the message parser. Words + of *TEXT MAY contain characters from character sets other than ISO- + 8859-1 [22] only when encoded according to the rules of RFC 2047 + [14]. + + TEXT = + + A CRLF is allowed in the definition of TEXT only as part of a header + field continuation. It is expected that the folding LWS will be + replaced with a single SP before interpretation of the TEXT value. + + Hexadecimal numeric characters are used in several protocol elements. + + HEX = "A" | "B" | "C" | "D" | "E" | "F" + | "a" | "b" | "c" | "d" | "e" | "f" | DIGIT + + + + + +Fielding, et al. Standards Track [Page 16] + +RFC 2616 HTTP/1.1 June 1999 + + + Many HTTP/1.1 header field values consist of words separated by LWS + or special characters. These special characters MUST be in a quoted + string to be used within a parameter value (as defined in section + 3.6). + + token = 1* + separators = "(" | ")" | "<" | ">" | "@" + | "," | ";" | ":" | "\" | <"> + | "/" | "[" | "]" | "?" | "=" + | "{" | "}" | SP | HT + + Comments can be included in some HTTP header fields by surrounding + the comment text with parentheses. Comments are only allowed in + fields containing "comment" as part of their field value definition. + In all other fields, parentheses are considered part of the field + value. + + comment = "(" *( ctext | quoted-pair | comment ) ")" + ctext = + + A string of text is parsed as a single word if it is quoted using + double-quote marks. + + quoted-string = ( <"> *(qdtext | quoted-pair ) <"> ) + qdtext = > + + The backslash character ("\") MAY be used as a single-character + quoting mechanism only within quoted-string and comment constructs. + + quoted-pair = "\" CHAR + +3 Protocol Parameters + +3.1 HTTP Version + + HTTP uses a "." numbering scheme to indicate versions + of the protocol. The protocol versioning policy is intended to allow + the sender to indicate the format of a message and its capacity for + understanding further HTTP communication, rather than the features + obtained via that communication. No change is made to the version + number for the addition of message components which do not affect + communication behavior or which only add to extensible field values. + The number is incremented when the changes made to the + protocol add features which do not change the general message parsing + algorithm, but which may add to the message semantics and imply + additional capabilities of the sender. The number is + incremented when the format of a message within the protocol is + changed. See RFC 2145 [36] for a fuller explanation. + + + +Fielding, et al. Standards Track [Page 17] + +RFC 2616 HTTP/1.1 June 1999 + + + The version of an HTTP message is indicated by an HTTP-Version field + in the first line of the message. + + HTTP-Version = "HTTP" "/" 1*DIGIT "." 1*DIGIT + + Note that the major and minor numbers MUST be treated as separate + integers and that each MAY be incremented higher than a single digit. + Thus, HTTP/2.4 is a lower version than HTTP/2.13, which in turn is + lower than HTTP/12.3. Leading zeros MUST be ignored by recipients and + MUST NOT be sent. + + An application that sends a request or response message that includes + HTTP-Version of "HTTP/1.1" MUST be at least conditionally compliant + with this specification. Applications that are at least conditionally + compliant with this specification SHOULD use an HTTP-Version of + "HTTP/1.1" in their messages, and MUST do so for any message that is + not compatible with HTTP/1.0. For more details on when to send + specific HTTP-Version values, see RFC 2145 [36]. + + The HTTP version of an application is the highest HTTP version for + which the application is at least conditionally compliant. + + Proxy and gateway applications need to be careful when forwarding + messages in protocol versions different from that of the application. + Since the protocol version indicates the protocol capability of the + sender, a proxy/gateway MUST NOT send a message with a version + indicator which is greater than its actual version. If a higher + version request is received, the proxy/gateway MUST either downgrade + the request version, or respond with an error, or switch to tunnel + behavior. + + Due to interoperability problems with HTTP/1.0 proxies discovered + since the publication of RFC 2068[33], caching proxies MUST, gateways + MAY, and tunnels MUST NOT upgrade the request to the highest version + they support. The proxy/gateway's response to that request MUST be in + the same major version as the request. + + Note: Converting between versions of HTTP may involve modification + of header fields required or forbidden by the versions involved. + +3.2 Uniform Resource Identifiers + + URIs have been known by many names: WWW addresses, Universal Document + Identifiers, Universal Resource Identifiers [3], and finally the + combination of Uniform Resource Locators (URL) [4] and Names (URN) + [20]. As far as HTTP is concerned, Uniform Resource Identifiers are + simply formatted strings which identify--via name, location, or any + other characteristic--a resource. + + + +Fielding, et al. Standards Track [Page 18] + +RFC 2616 HTTP/1.1 June 1999 + + +3.2.1 General Syntax + + URIs in HTTP can be represented in absolute form or relative to some + known base URI [11], depending upon the context of their use. The two + forms are differentiated by the fact that absolute URIs always begin + with a scheme name followed by a colon. For definitive information on + URL syntax and semantics, see "Uniform Resource Identifiers (URI): + Generic Syntax and Semantics," RFC 2396 [42] (which replaces RFCs + 1738 [4] and RFC 1808 [11]). This specification adopts the + definitions of "URI-reference", "absoluteURI", "relativeURI", "port", + "host","abs_path", "rel_path", and "authority" from that + specification. + + The HTTP protocol does not place any a priori limit on the length of + a URI. Servers MUST be able to handle the URI of any resource they + serve, and SHOULD be able to handle URIs of unbounded length if they + provide GET-based forms that could generate such URIs. A server + SHOULD return 414 (Request-URI Too Long) status if a URI is longer + than the server can handle (see section 10.4.15). + + Note: Servers ought to be cautious about depending on URI lengths + above 255 bytes, because some older client or proxy + implementations might not properly support these lengths. + +3.2.2 http URL + + The "http" scheme is used to locate network resources via the HTTP + protocol. This section defines the scheme-specific syntax and + semantics for http URLs. + + http_URL = "http:" "//" host [ ":" port ] [ abs_path [ "?" query ]] + + If the port is empty or not given, port 80 is assumed. The semantics + are that the identified resource is located at the server listening + for TCP connections on that port of that host, and the Request-URI + for the resource is abs_path (section 5.1.2). The use of IP addresses + in URLs SHOULD be avoided whenever possible (see RFC 1900 [24]). If + the abs_path is not present in the URL, it MUST be given as "/" when + used as a Request-URI for a resource (section 5.1.2). If a proxy + receives a host name which is not a fully qualified domain name, it + MAY add its domain to the host name it received. If a proxy receives + a fully qualified domain name, the proxy MUST NOT change the host + name. + + + + + + + + +Fielding, et al. Standards Track [Page 19] + +RFC 2616 HTTP/1.1 June 1999 + + +3.2.3 URI Comparison + + When comparing two URIs to decide if they match or not, a client + SHOULD use a case-sensitive octet-by-octet comparison of the entire + URIs, with these exceptions: + + - A port that is empty or not given is equivalent to the default + port for that URI-reference; + + - Comparisons of host names MUST be case-insensitive; + + - Comparisons of scheme names MUST be case-insensitive; + + - An empty abs_path is equivalent to an abs_path of "/". + + Characters other than those in the "reserved" and "unsafe" sets (see + RFC 2396 [42]) are equivalent to their ""%" HEX HEX" encoding. + + For example, the following three URIs are equivalent: + + http://abc.com:80/~smith/home.html + http://ABC.com/%7Esmith/home.html + http://ABC.com:/%7esmith/home.html + +3.3 Date/Time Formats + +3.3.1 Full Date + + HTTP applications have historically allowed three different formats + for the representation of date/time stamps: + + Sun, 06 Nov 1994 08:49:37 GMT ; RFC 822, updated by RFC 1123 + Sunday, 06-Nov-94 08:49:37 GMT ; RFC 850, obsoleted by RFC 1036 + Sun Nov 6 08:49:37 1994 ; ANSI C's asctime() format + + The first format is preferred as an Internet standard and represents + a fixed-length subset of that defined by RFC 1123 [8] (an update to + RFC 822 [9]). The second format is in common use, but is based on the + obsolete RFC 850 [12] date format and lacks a four-digit year. + HTTP/1.1 clients and servers that parse the date value MUST accept + all three formats (for compatibility with HTTP/1.0), though they MUST + only generate the RFC 1123 format for representing HTTP-date values + in header fields. See section 19.3 for further information. + + Note: Recipients of date values are encouraged to be robust in + accepting date values that may have been sent by non-HTTP + applications, as is sometimes the case when retrieving or posting + messages via proxies/gateways to SMTP or NNTP. + + + +Fielding, et al. Standards Track [Page 20] + +RFC 2616 HTTP/1.1 June 1999 + + + All HTTP date/time stamps MUST be represented in Greenwich Mean Time + (GMT), without exception. For the purposes of HTTP, GMT is exactly + equal to UTC (Coordinated Universal Time). This is indicated in the + first two formats by the inclusion of "GMT" as the three-letter + abbreviation for time zone, and MUST be assumed when reading the + asctime format. HTTP-date is case sensitive and MUST NOT include + additional LWS beyond that specifically included as SP in the + grammar. + + HTTP-date = rfc1123-date | rfc850-date | asctime-date + rfc1123-date = wkday "," SP date1 SP time SP "GMT" + rfc850-date = weekday "," SP date2 SP time SP "GMT" + asctime-date = wkday SP date3 SP time SP 4DIGIT + date1 = 2DIGIT SP month SP 4DIGIT + ; day month year (e.g., 02 Jun 1982) + date2 = 2DIGIT "-" month "-" 2DIGIT + ; day-month-year (e.g., 02-Jun-82) + date3 = month SP ( 2DIGIT | ( SP 1DIGIT )) + ; month day (e.g., Jun 2) + time = 2DIGIT ":" 2DIGIT ":" 2DIGIT + ; 00:00:00 - 23:59:59 + wkday = "Mon" | "Tue" | "Wed" + | "Thu" | "Fri" | "Sat" | "Sun" + weekday = "Monday" | "Tuesday" | "Wednesday" + | "Thursday" | "Friday" | "Saturday" | "Sunday" + month = "Jan" | "Feb" | "Mar" | "Apr" + | "May" | "Jun" | "Jul" | "Aug" + | "Sep" | "Oct" | "Nov" | "Dec" + + Note: HTTP requirements for the date/time stamp format apply only + to their usage within the protocol stream. Clients and servers are + not required to use these formats for user presentation, request + logging, etc. + +3.3.2 Delta Seconds + + Some HTTP header fields allow a time value to be specified as an + integer number of seconds, represented in decimal, after the time + that the message was received. + + delta-seconds = 1*DIGIT + +3.4 Character Sets + + HTTP uses the same definition of the term "character set" as that + described for MIME: + + + + + +Fielding, et al. Standards Track [Page 21] + +RFC 2616 HTTP/1.1 June 1999 + + + The term "character set" is used in this document to refer to a + method used with one or more tables to convert a sequence of octets + into a sequence of characters. Note that unconditional conversion in + the other direction is not required, in that not all characters may + be available in a given character set and a character set may provide + more than one sequence of octets to represent a particular character. + This definition is intended to allow various kinds of character + encoding, from simple single-table mappings such as US-ASCII to + complex table switching methods such as those that use ISO-2022's + techniques. However, the definition associated with a MIME character + set name MUST fully specify the mapping to be performed from octets + to characters. In particular, use of external profiling information + to determine the exact mapping is not permitted. + + Note: This use of the term "character set" is more commonly + referred to as a "character encoding." However, since HTTP and + MIME share the same registry, it is important that the terminology + also be shared. + + HTTP character sets are identified by case-insensitive tokens. The + complete set of tokens is defined by the IANA Character Set registry + [19]. + + charset = token + + Although HTTP allows an arbitrary token to be used as a charset + value, any token that has a predefined value within the IANA + Character Set registry [19] MUST represent the character set defined + by that registry. Applications SHOULD limit their use of character + sets to those defined by the IANA registry. + + Implementors should be aware of IETF character set requirements [38] + [41]. + +3.4.1 Missing Charset + + Some HTTP/1.0 software has interpreted a Content-Type header without + charset parameter incorrectly to mean "recipient should guess." + Senders wishing to defeat this behavior MAY include a charset + parameter even when the charset is ISO-8859-1 and SHOULD do so when + it is known that it will not confuse the recipient. + + Unfortunately, some older HTTP/1.0 clients did not deal properly with + an explicit charset parameter. HTTP/1.1 recipients MUST respect the + charset label provided by the sender; and those user agents that have + a provision to "guess" a charset MUST use the charset from the + + + + + +Fielding, et al. Standards Track [Page 22] + +RFC 2616 HTTP/1.1 June 1999 + + + content-type field if they support that charset, rather than the + recipient's preference, when initially displaying a document. See + section 3.7.1. + +3.5 Content Codings + + Content coding values indicate an encoding transformation that has + been or can be applied to an entity. Content codings are primarily + used to allow a document to be compressed or otherwise usefully + transformed without losing the identity of its underlying media type + and without loss of information. Frequently, the entity is stored in + coded form, transmitted directly, and only decoded by the recipient. + + content-coding = token + + All content-coding values are case-insensitive. HTTP/1.1 uses + content-coding values in the Accept-Encoding (section 14.3) and + Content-Encoding (section 14.11) header fields. Although the value + describes the content-coding, what is more important is that it + indicates what decoding mechanism will be required to remove the + encoding. + + The Internet Assigned Numbers Authority (IANA) acts as a registry for + content-coding value tokens. Initially, the registry contains the + following tokens: + + gzip An encoding format produced by the file compression program + "gzip" (GNU zip) as described in RFC 1952 [25]. This format is a + Lempel-Ziv coding (LZ77) with a 32 bit CRC. + + compress + The encoding format produced by the common UNIX file compression + program "compress". This format is an adaptive Lempel-Ziv-Welch + coding (LZW). + + Use of program names for the identification of encoding formats + is not desirable and is discouraged for future encodings. Their + use here is representative of historical practice, not good + design. For compatibility with previous implementations of HTTP, + applications SHOULD consider "x-gzip" and "x-compress" to be + equivalent to "gzip" and "compress" respectively. + + deflate + The "zlib" format defined in RFC 1950 [31] in combination with + the "deflate" compression mechanism described in RFC 1951 [29]. + + + + + + +Fielding, et al. Standards Track [Page 23] + +RFC 2616 HTTP/1.1 June 1999 + + + identity + The default (identity) encoding; the use of no transformation + whatsoever. This content-coding is used only in the Accept- + Encoding header, and SHOULD NOT be used in the Content-Encoding + header. + + New content-coding value tokens SHOULD be registered; to allow + interoperability between clients and servers, specifications of the + content coding algorithms needed to implement a new value SHOULD be + publicly available and adequate for independent implementation, and + conform to the purpose of content coding defined in this section. + +3.6 Transfer Codings + + Transfer-coding values are used to indicate an encoding + transformation that has been, can be, or may need to be applied to an + entity-body in order to ensure "safe transport" through the network. + This differs from a content coding in that the transfer-coding is a + property of the message, not of the original entity. + + transfer-coding = "chunked" | transfer-extension + transfer-extension = token *( ";" parameter ) + + Parameters are in the form of attribute/value pairs. + + parameter = attribute "=" value + attribute = token + value = token | quoted-string + + All transfer-coding values are case-insensitive. HTTP/1.1 uses + transfer-coding values in the TE header field (section 14.39) and in + the Transfer-Encoding header field (section 14.41). + + Whenever a transfer-coding is applied to a message-body, the set of + transfer-codings MUST include "chunked", unless the message is + terminated by closing the connection. When the "chunked" transfer- + coding is used, it MUST be the last transfer-coding applied to the + message-body. The "chunked" transfer-coding MUST NOT be applied more + than once to a message-body. These rules allow the recipient to + determine the transfer-length of the message (section 4.4). + + Transfer-codings are analogous to the Content-Transfer-Encoding + values of MIME [7], which were designed to enable safe transport of + binary data over a 7-bit transport service. However, safe transport + has a different focus for an 8bit-clean transfer protocol. In HTTP, + the only unsafe characteristic of message-bodies is the difficulty in + determining the exact body length (section 7.2.2), or the desire to + encrypt data over a shared transport. + + + +Fielding, et al. Standards Track [Page 24] + +RFC 2616 HTTP/1.1 June 1999 + + + The Internet Assigned Numbers Authority (IANA) acts as a registry for + transfer-coding value tokens. Initially, the registry contains the + following tokens: "chunked" (section 3.6.1), "identity" (section + 3.6.2), "gzip" (section 3.5), "compress" (section 3.5), and "deflate" + (section 3.5). + + New transfer-coding value tokens SHOULD be registered in the same way + as new content-coding value tokens (section 3.5). + + A server which receives an entity-body with a transfer-coding it does + not understand SHOULD return 501 (Unimplemented), and close the + connection. A server MUST NOT send transfer-codings to an HTTP/1.0 + client. + +3.6.1 Chunked Transfer Coding + + The chunked encoding modifies the body of a message in order to + transfer it as a series of chunks, each with its own size indicator, + followed by an OPTIONAL trailer containing entity-header fields. This + allows dynamically produced content to be transferred along with the + information necessary for the recipient to verify that it has + received the full message. + + Chunked-Body = *chunk + last-chunk + trailer + CRLF + + chunk = chunk-size [ chunk-extension ] CRLF + chunk-data CRLF + chunk-size = 1*HEX + last-chunk = 1*("0") [ chunk-extension ] CRLF + + chunk-extension= *( ";" chunk-ext-name [ "=" chunk-ext-val ] ) + chunk-ext-name = token + chunk-ext-val = token | quoted-string + chunk-data = chunk-size(OCTET) + trailer = *(entity-header CRLF) + + The chunk-size field is a string of hex digits indicating the size of + the chunk. The chunked encoding is ended by any chunk whose size is + zero, followed by the trailer, which is terminated by an empty line. + + The trailer allows the sender to include additional HTTP header + fields at the end of the message. The Trailer header field can be + used to indicate which header fields are included in a trailer (see + section 14.40). + + + + +Fielding, et al. Standards Track [Page 25] + +RFC 2616 HTTP/1.1 June 1999 + + + A server using chunked transfer-coding in a response MUST NOT use the + trailer for any header fields unless at least one of the following is + true: + + a)the request included a TE header field that indicates "trailers" is + acceptable in the transfer-coding of the response, as described in + section 14.39; or, + + b)the server is the origin server for the response, the trailer + fields consist entirely of optional metadata, and the recipient + could use the message (in a manner acceptable to the origin server) + without receiving this metadata. In other words, the origin server + is willing to accept the possibility that the trailer fields might + be silently discarded along the path to the client. + + This requirement prevents an interoperability failure when the + message is being received by an HTTP/1.1 (or later) proxy and + forwarded to an HTTP/1.0 recipient. It avoids a situation where + compliance with the protocol would have necessitated a possibly + infinite buffer on the proxy. + + An example process for decoding a Chunked-Body is presented in + appendix 19.4.6. + + All HTTP/1.1 applications MUST be able to receive and decode the + "chunked" transfer-coding, and MUST ignore chunk-extension extensions + they do not understand. + +3.7 Media Types + + HTTP uses Internet Media Types [17] in the Content-Type (section + 14.17) and Accept (section 14.1) header fields in order to provide + open and extensible data typing and type negotiation. + + media-type = type "/" subtype *( ";" parameter ) + type = token + subtype = token + + Parameters MAY follow the type/subtype in the form of attribute/value + pairs (as defined in section 3.6). + + The type, subtype, and parameter attribute names are case- + insensitive. Parameter values might or might not be case-sensitive, + depending on the semantics of the parameter name. Linear white space + (LWS) MUST NOT be used between the type and subtype, nor between an + attribute and its value. The presence or absence of a parameter might + be significant to the processing of a media-type, depending on its + definition within the media type registry. + + + +Fielding, et al. Standards Track [Page 26] + +RFC 2616 HTTP/1.1 June 1999 + + + Note that some older HTTP applications do not recognize media type + parameters. When sending data to older HTTP applications, + implementations SHOULD only use media type parameters when they are + required by that type/subtype definition. + + Media-type values are registered with the Internet Assigned Number + Authority (IANA [19]). The media type registration process is + outlined in RFC 1590 [17]. Use of non-registered media types is + discouraged. + +3.7.1 Canonicalization and Text Defaults + + Internet media types are registered with a canonical form. An + entity-body transferred via HTTP messages MUST be represented in the + appropriate canonical form prior to its transmission except for + "text" types, as defined in the next paragraph. + + When in canonical form, media subtypes of the "text" type use CRLF as + the text line break. HTTP relaxes this requirement and allows the + transport of text media with plain CR or LF alone representing a line + break when it is done consistently for an entire entity-body. HTTP + applications MUST accept CRLF, bare CR, and bare LF as being + representative of a line break in text media received via HTTP. In + addition, if the text is represented in a character set that does not + use octets 13 and 10 for CR and LF respectively, as is the case for + some multi-byte character sets, HTTP allows the use of whatever octet + sequences are defined by that character set to represent the + equivalent of CR and LF for line breaks. This flexibility regarding + line breaks applies only to text media in the entity-body; a bare CR + or LF MUST NOT be substituted for CRLF within any of the HTTP control + structures (such as header fields and multipart boundaries). + + If an entity-body is encoded with a content-coding, the underlying + data MUST be in a form defined above prior to being encoded. + + The "charset" parameter is used with some media types to define the + character set (section 3.4) of the data. When no explicit charset + parameter is provided by the sender, media subtypes of the "text" + type are defined to have a default charset value of "ISO-8859-1" when + received via HTTP. Data in character sets other than "ISO-8859-1" or + its subsets MUST be labeled with an appropriate charset value. See + section 3.4.1 for compatibility problems. + +3.7.2 Multipart Types + + MIME provides for a number of "multipart" types -- encapsulations of + one or more entities within a single message-body. All multipart + types share a common syntax, as defined in section 5.1.1 of RFC 2046 + + + +Fielding, et al. Standards Track [Page 27] + +RFC 2616 HTTP/1.1 June 1999 + + + [40], and MUST include a boundary parameter as part of the media type + value. The message body is itself a protocol element and MUST + therefore use only CRLF to represent line breaks between body-parts. + Unlike in RFC 2046, the epilogue of any multipart message MUST be + empty; HTTP applications MUST NOT transmit the epilogue (even if the + original multipart contains an epilogue). These restrictions exist in + order to preserve the self-delimiting nature of a multipart message- + body, wherein the "end" of the message-body is indicated by the + ending multipart boundary. + + In general, HTTP treats a multipart message-body no differently than + any other media type: strictly as payload. The one exception is the + "multipart/byteranges" type (appendix 19.2) when it appears in a 206 + (Partial Content) response, which will be interpreted by some HTTP + caching mechanisms as described in sections 13.5.4 and 14.16. In all + other cases, an HTTP user agent SHOULD follow the same or similar + behavior as a MIME user agent would upon receipt of a multipart type. + The MIME header fields within each body-part of a multipart message- + body do not have any significance to HTTP beyond that defined by + their MIME semantics. + + In general, an HTTP user agent SHOULD follow the same or similar + behavior as a MIME user agent would upon receipt of a multipart type. + If an application receives an unrecognized multipart subtype, the + application MUST treat it as being equivalent to "multipart/mixed". + + Note: The "multipart/form-data" type has been specifically defined + for carrying form data suitable for processing via the POST + request method, as described in RFC 1867 [15]. + +3.8 Product Tokens + + Product tokens are used to allow communicating applications to + identify themselves by software name and version. Most fields using + product tokens also allow sub-products which form a significant part + of the application to be listed, separated by white space. By + convention, the products are listed in order of their significance + for identifying the application. + + product = token ["/" product-version] + product-version = token + + Examples: + + User-Agent: CERN-LineMode/2.15 libwww/2.17b3 + Server: Apache/0.8.4 + + + + + +Fielding, et al. Standards Track [Page 28] + +RFC 2616 HTTP/1.1 June 1999 + + + Product tokens SHOULD be short and to the point. They MUST NOT be + used for advertising or other non-essential information. Although any + token character MAY appear in a product-version, this token SHOULD + only be used for a version identifier (i.e., successive versions of + the same product SHOULD only differ in the product-version portion of + the product value). + +3.9 Quality Values + + HTTP content negotiation (section 12) uses short "floating point" + numbers to indicate the relative importance ("weight") of various + negotiable parameters. A weight is normalized to a real number in + the range 0 through 1, where 0 is the minimum and 1 the maximum + value. If a parameter has a quality value of 0, then content with + this parameter is `not acceptable' for the client. HTTP/1.1 + applications MUST NOT generate more than three digits after the + decimal point. User configuration of these values SHOULD also be + limited in this fashion. + + qvalue = ( "0" [ "." 0*3DIGIT ] ) + | ( "1" [ "." 0*3("0") ] ) + + "Quality values" is a misnomer, since these values merely represent + relative degradation in desired quality. + +3.10 Language Tags + + A language tag identifies a natural language spoken, written, or + otherwise conveyed by human beings for communication of information + to other human beings. Computer languages are explicitly excluded. + HTTP uses language tags within the Accept-Language and Content- + Language fields. + + The syntax and registry of HTTP language tags is the same as that + defined by RFC 1766 [1]. In summary, a language tag is composed of 1 + or more parts: A primary language tag and a possibly empty series of + subtags: + + language-tag = primary-tag *( "-" subtag ) + primary-tag = 1*8ALPHA + subtag = 1*8ALPHA + + White space is not allowed within the tag and all tags are case- + insensitive. The name space of language tags is administered by the + IANA. Example tags include: + + en, en-US, en-cockney, i-cherokee, x-pig-latin + + + + +Fielding, et al. Standards Track [Page 29] + +RFC 2616 HTTP/1.1 June 1999 + + + where any two-letter primary-tag is an ISO-639 language abbreviation + and any two-letter initial subtag is an ISO-3166 country code. (The + last three tags above are not registered tags; all but the last are + examples of tags which could be registered in future.) + +3.11 Entity Tags + + Entity tags are used for comparing two or more entities from the same + requested resource. HTTP/1.1 uses entity tags in the ETag (section + 14.19), If-Match (section 14.24), If-None-Match (section 14.26), and + If-Range (section 14.27) header fields. The definition of how they + are used and compared as cache validators is in section 13.3.3. An + entity tag consists of an opaque quoted string, possibly prefixed by + a weakness indicator. + + entity-tag = [ weak ] opaque-tag + weak = "W/" + opaque-tag = quoted-string + + A "strong entity tag" MAY be shared by two entities of a resource + only if they are equivalent by octet equality. + + A "weak entity tag," indicated by the "W/" prefix, MAY be shared by + two entities of a resource only if the entities are equivalent and + could be substituted for each other with no significant change in + semantics. A weak entity tag can only be used for weak comparison. + + An entity tag MUST be unique across all versions of all entities + associated with a particular resource. A given entity tag value MAY + be used for entities obtained by requests on different URIs. The use + of the same entity tag value in conjunction with entities obtained by + requests on different URIs does not imply the equivalence of those + entities. + +3.12 Range Units + + HTTP/1.1 allows a client to request that only part (a range of) the + response entity be included within the response. HTTP/1.1 uses range + units in the Range (section 14.35) and Content-Range (section 14.16) + header fields. An entity can be broken down into subranges according + to various structural units. + + range-unit = bytes-unit | other-range-unit + bytes-unit = "bytes" + other-range-unit = token + + The only range unit defined by HTTP/1.1 is "bytes". HTTP/1.1 + implementations MAY ignore ranges specified using other units. + + + +Fielding, et al. Standards Track [Page 30] + +RFC 2616 HTTP/1.1 June 1999 + + + HTTP/1.1 has been designed to allow implementations of applications + that do not depend on knowledge of ranges. + +4 HTTP Message + +4.1 Message Types + + HTTP messages consist of requests from client to server and responses + from server to client. + + HTTP-message = Request | Response ; HTTP/1.1 messages + + Request (section 5) and Response (section 6) messages use the generic + message format of RFC 822 [9] for transferring entities (the payload + of the message). Both types of message consist of a start-line, zero + or more header fields (also known as "headers"), an empty line (i.e., + a line with nothing preceding the CRLF) indicating the end of the + header fields, and possibly a message-body. + + generic-message = start-line + *(message-header CRLF) + CRLF + [ message-body ] + start-line = Request-Line | Status-Line + + In the interest of robustness, servers SHOULD ignore any empty + line(s) received where a Request-Line is expected. In other words, if + the server is reading the protocol stream at the beginning of a + message and receives a CRLF first, it should ignore the CRLF. + + Certain buggy HTTP/1.0 client implementations generate extra CRLF's + after a POST request. To restate what is explicitly forbidden by the + BNF, an HTTP/1.1 client MUST NOT preface or follow a request with an + extra CRLF. + +4.2 Message Headers + + HTTP header fields, which include general-header (section 4.5), + request-header (section 5.3), response-header (section 6.2), and + entity-header (section 7.1) fields, follow the same generic format as + that given in Section 3.1 of RFC 822 [9]. Each header field consists + of a name followed by a colon (":") and the field value. Field names + are case-insensitive. The field value MAY be preceded by any amount + of LWS, though a single SP is preferred. Header fields can be + extended over multiple lines by preceding each extra line with at + least one SP or HT. Applications ought to follow "common form", where + one is known or indicated, when generating HTTP constructs, since + there might exist some implementations that fail to accept anything + + + +Fielding, et al. Standards Track [Page 31] + +RFC 2616 HTTP/1.1 June 1999 + + + beyond the common forms. + + message-header = field-name ":" [ field-value ] + field-name = token + field-value = *( field-content | LWS ) + field-content = + + The field-content does not include any leading or trailing LWS: + linear white space occurring before the first non-whitespace + character of the field-value or after the last non-whitespace + character of the field-value. Such leading or trailing LWS MAY be + removed without changing the semantics of the field value. Any LWS + that occurs between field-content MAY be replaced with a single SP + before interpreting the field value or forwarding the message + downstream. + + The order in which header fields with differing field names are + received is not significant. However, it is "good practice" to send + general-header fields first, followed by request-header or response- + header fields, and ending with the entity-header fields. + + Multiple message-header fields with the same field-name MAY be + present in a message if and only if the entire field-value for that + header field is defined as a comma-separated list [i.e., #(values)]. + It MUST be possible to combine the multiple header fields into one + "field-name: field-value" pair, without changing the semantics of the + message, by appending each subsequent field-value to the first, each + separated by a comma. The order in which header fields with the same + field-name are received is therefore significant to the + interpretation of the combined field value, and thus a proxy MUST NOT + change the order of these field values when a message is forwarded. + +4.3 Message Body + + The message-body (if any) of an HTTP message is used to carry the + entity-body associated with the request or response. The message-body + differs from the entity-body only when a transfer-coding has been + applied, as indicated by the Transfer-Encoding header field (section + 14.41). + + message-body = entity-body + | + + Transfer-Encoding MUST be used to indicate any transfer-codings + applied by an application to ensure safe and proper transfer of the + message. Transfer-Encoding is a property of the message, not of the + + + +Fielding, et al. Standards Track [Page 32] + +RFC 2616 HTTP/1.1 June 1999 + + + entity, and thus MAY be added or removed by any application along the + request/response chain. (However, section 3.6 places restrictions on + when certain transfer-codings may be used.) + + The rules for when a message-body is allowed in a message differ for + requests and responses. + + The presence of a message-body in a request is signaled by the + inclusion of a Content-Length or Transfer-Encoding header field in + the request's message-headers. A message-body MUST NOT be included in + a request if the specification of the request method (section 5.1.1) + does not allow sending an entity-body in requests. A server SHOULD + read and forward a message-body on any request; if the request method + does not include defined semantics for an entity-body, then the + message-body SHOULD be ignored when handling the request. + + For response messages, whether or not a message-body is included with + a message is dependent on both the request method and the response + status code (section 6.1.1). All responses to the HEAD request method + MUST NOT include a message-body, even though the presence of entity- + header fields might lead one to believe they do. All 1xx + (informational), 204 (no content), and 304 (not modified) responses + MUST NOT include a message-body. All other responses do include a + message-body, although it MAY be of zero length. + +4.4 Message Length + + The transfer-length of a message is the length of the message-body as + it appears in the message; that is, after any transfer-codings have + been applied. When a message-body is included with a message, the + transfer-length of that body is determined by one of the following + (in order of precedence): + + 1.Any response message which "MUST NOT" include a message-body (such + as the 1xx, 204, and 304 responses and any response to a HEAD + request) is always terminated by the first empty line after the + header fields, regardless of the entity-header fields present in + the message. + + 2.If a Transfer-Encoding header field (section 14.41) is present and + has any value other than "identity", then the transfer-length is + defined by use of the "chunked" transfer-coding (section 3.6), + unless the message is terminated by closing the connection. + + 3.If a Content-Length header field (section 14.13) is present, its + decimal value in OCTETs represents both the entity-length and the + transfer-length. The Content-Length header field MUST NOT be sent + if these two lengths are different (i.e., if a Transfer-Encoding + + + +Fielding, et al. Standards Track [Page 33] + +RFC 2616 HTTP/1.1 June 1999 + + + header field is present). If a message is received with both a + Transfer-Encoding header field and a Content-Length header field, + the latter MUST be ignored. + + 4.If the message uses the media type "multipart/byteranges", and the + ransfer-length is not otherwise specified, then this self- + elimiting media type defines the transfer-length. This media type + UST NOT be used unless the sender knows that the recipient can arse + it; the presence in a request of a Range header with ultiple byte- + range specifiers from a 1.1 client implies that the lient can parse + multipart/byteranges responses. + + A range header might be forwarded by a 1.0 proxy that does not + understand multipart/byteranges; in this case the server MUST + delimit the message using methods defined in items 1,3 or 5 of + this section. + + 5.By the server closing the connection. (Closing the connection + cannot be used to indicate the end of a request body, since that + would leave no possibility for the server to send back a response.) + + For compatibility with HTTP/1.0 applications, HTTP/1.1 requests + containing a message-body MUST include a valid Content-Length header + field unless the server is known to be HTTP/1.1 compliant. If a + request contains a message-body and a Content-Length is not given, + the server SHOULD respond with 400 (bad request) if it cannot + determine the length of the message, or with 411 (length required) if + it wishes to insist on receiving a valid Content-Length. + + All HTTP/1.1 applications that receive entities MUST accept the + "chunked" transfer-coding (section 3.6), thus allowing this mechanism + to be used for messages when the message length cannot be determined + in advance. + + Messages MUST NOT include both a Content-Length header field and a + non-identity transfer-coding. If the message does include a non- + identity transfer-coding, the Content-Length MUST be ignored. + + When a Content-Length is given in a message where a message-body is + allowed, its field value MUST exactly match the number of OCTETs in + the message-body. HTTP/1.1 user agents MUST notify the user when an + invalid length is received and detected. + +4.5 General Header Fields + + There are a few header fields which have general applicability for + both request and response messages, but which do not apply to the + entity being transferred. These header fields apply only to the + + + +Fielding, et al. Standards Track [Page 34] + +RFC 2616 HTTP/1.1 June 1999 + + + message being transmitted. + + general-header = Cache-Control ; Section 14.9 + | Connection ; Section 14.10 + | Date ; Section 14.18 + | Pragma ; Section 14.32 + | Trailer ; Section 14.40 + | Transfer-Encoding ; Section 14.41 + | Upgrade ; Section 14.42 + | Via ; Section 14.45 + | Warning ; Section 14.46 + + General-header field names can be extended reliably only in + combination with a change in the protocol version. However, new or + experimental header fields may be given the semantics of general + header fields if all parties in the communication recognize them to + be general-header fields. Unrecognized header fields are treated as + entity-header fields. + +5 Request + + A request message from a client to a server includes, within the + first line of that message, the method to be applied to the resource, + the identifier of the resource, and the protocol version in use. + + Request = Request-Line ; Section 5.1 + *(( general-header ; Section 4.5 + | request-header ; Section 5.3 + | entity-header ) CRLF) ; Section 7.1 + CRLF + [ message-body ] ; Section 4.3 + +5.1 Request-Line + + The Request-Line begins with a method token, followed by the + Request-URI and the protocol version, and ending with CRLF. The + elements are separated by SP characters. No CR or LF is allowed + except in the final CRLF sequence. + + Request-Line = Method SP Request-URI SP HTTP-Version CRLF + + + + + + + + + + + +Fielding, et al. Standards Track [Page 35] + +RFC 2616 HTTP/1.1 June 1999 + + +5.1.1 Method + + The Method token indicates the method to be performed on the + resource identified by the Request-URI. The method is case-sensitive. + + Method = "OPTIONS" ; Section 9.2 + | "GET" ; Section 9.3 + | "HEAD" ; Section 9.4 + | "POST" ; Section 9.5 + | "PUT" ; Section 9.6 + | "DELETE" ; Section 9.7 + | "TRACE" ; Section 9.8 + | "CONNECT" ; Section 9.9 + | extension-method + extension-method = token + + The list of methods allowed by a resource can be specified in an + Allow header field (section 14.7). The return code of the response + always notifies the client whether a method is currently allowed on a + resource, since the set of allowed methods can change dynamically. An + origin server SHOULD return the status code 405 (Method Not Allowed) + if the method is known by the origin server but not allowed for the + requested resource, and 501 (Not Implemented) if the method is + unrecognized or not implemented by the origin server. The methods GET + and HEAD MUST be supported by all general-purpose servers. All other + methods are OPTIONAL; however, if the above methods are implemented, + they MUST be implemented with the same semantics as those specified + in section 9. + +5.1.2 Request-URI + + The Request-URI is a Uniform Resource Identifier (section 3.2) and + identifies the resource upon which to apply the request. + + Request-URI = "*" | absoluteURI | abs_path | authority + + The four options for Request-URI are dependent on the nature of the + request. The asterisk "*" means that the request does not apply to a + particular resource, but to the server itself, and is only allowed + when the method used does not necessarily apply to a resource. One + example would be + + OPTIONS * HTTP/1.1 + + The absoluteURI form is REQUIRED when the request is being made to a + proxy. The proxy is requested to forward the request or service it + from a valid cache, and return the response. Note that the proxy MAY + forward the request on to another proxy or directly to the server + + + +Fielding, et al. Standards Track [Page 36] + +RFC 2616 HTTP/1.1 June 1999 + + + specified by the absoluteURI. In order to avoid request loops, a + proxy MUST be able to recognize all of its server names, including + any aliases, local variations, and the numeric IP address. An example + Request-Line would be: + + GET http://www.w3.org/pub/WWW/TheProject.html HTTP/1.1 + + To allow for transition to absoluteURIs in all requests in future + versions of HTTP, all HTTP/1.1 servers MUST accept the absoluteURI + form in requests, even though HTTP/1.1 clients will only generate + them in requests to proxies. + + The authority form is only used by the CONNECT method (section 9.9). + + The most common form of Request-URI is that used to identify a + resource on an origin server or gateway. In this case the absolute + path of the URI MUST be transmitted (see section 3.2.1, abs_path) as + the Request-URI, and the network location of the URI (authority) MUST + be transmitted in a Host header field. For example, a client wishing + to retrieve the resource above directly from the origin server would + create a TCP connection to port 80 of the host "www.w3.org" and send + the lines: + + GET /pub/WWW/TheProject.html HTTP/1.1 + Host: www.w3.org + + followed by the remainder of the Request. Note that the absolute path + cannot be empty; if none is present in the original URI, it MUST be + given as "/" (the server root). + + The Request-URI is transmitted in the format specified in section + 3.2.1. If the Request-URI is encoded using the "% HEX HEX" encoding + [42], the origin server MUST decode the Request-URI in order to + properly interpret the request. Servers SHOULD respond to invalid + Request-URIs with an appropriate status code. + + A transparent proxy MUST NOT rewrite the "abs_path" part of the + received Request-URI when forwarding it to the next inbound server, + except as noted above to replace a null abs_path with "/". + + Note: The "no rewrite" rule prevents the proxy from changing the + meaning of the request when the origin server is improperly using + a non-reserved URI character for a reserved purpose. Implementors + should be aware that some pre-HTTP/1.1 proxies have been known to + rewrite the Request-URI. + + + + + + +Fielding, et al. Standards Track [Page 37] + +RFC 2616 HTTP/1.1 June 1999 + + +5.2 The Resource Identified by a Request + + The exact resource identified by an Internet request is determined by + examining both the Request-URI and the Host header field. + + An origin server that does not allow resources to differ by the + requested host MAY ignore the Host header field value when + determining the resource identified by an HTTP/1.1 request. (But see + section 19.6.1.1 for other requirements on Host support in HTTP/1.1.) + + An origin server that does differentiate resources based on the host + requested (sometimes referred to as virtual hosts or vanity host + names) MUST use the following rules for determining the requested + resource on an HTTP/1.1 request: + + 1. If Request-URI is an absoluteURI, the host is part of the + Request-URI. Any Host header field value in the request MUST be + ignored. + + 2. If the Request-URI is not an absoluteURI, and the request includes + a Host header field, the host is determined by the Host header + field value. + + 3. If the host as determined by rule 1 or 2 is not a valid host on + the server, the response MUST be a 400 (Bad Request) error message. + + Recipients of an HTTP/1.0 request that lacks a Host header field MAY + attempt to use heuristics (e.g., examination of the URI path for + something unique to a particular host) in order to determine what + exact resource is being requested. + +5.3 Request Header Fields + + The request-header fields allow the client to pass additional + information about the request, and about the client itself, to the + server. These fields act as request modifiers, with semantics + equivalent to the parameters on a programming language method + invocation. + + request-header = Accept ; Section 14.1 + | Accept-Charset ; Section 14.2 + | Accept-Encoding ; Section 14.3 + | Accept-Language ; Section 14.4 + | Authorization ; Section 14.8 + | Expect ; Section 14.20 + | From ; Section 14.22 + | Host ; Section 14.23 + | If-Match ; Section 14.24 + + + +Fielding, et al. Standards Track [Page 38] + +RFC 2616 HTTP/1.1 June 1999 + + + | If-Modified-Since ; Section 14.25 + | If-None-Match ; Section 14.26 + | If-Range ; Section 14.27 + | If-Unmodified-Since ; Section 14.28 + | Max-Forwards ; Section 14.31 + | Proxy-Authorization ; Section 14.34 + | Range ; Section 14.35 + | Referer ; Section 14.36 + | TE ; Section 14.39 + | User-Agent ; Section 14.43 + + Request-header field names can be extended reliably only in + combination with a change in the protocol version. However, new or + experimental header fields MAY be given the semantics of request- + header fields if all parties in the communication recognize them to + be request-header fields. Unrecognized header fields are treated as + entity-header fields. + +6 Response + + After receiving and interpreting a request message, a server responds + with an HTTP response message. + + Response = Status-Line ; Section 6.1 + *(( general-header ; Section 4.5 + | response-header ; Section 6.2 + | entity-header ) CRLF) ; Section 7.1 + CRLF + [ message-body ] ; Section 7.2 + +6.1 Status-Line + + The first line of a Response message is the Status-Line, consisting + of the protocol version followed by a numeric status code and its + associated textual phrase, with each element separated by SP + characters. No CR or LF is allowed except in the final CRLF sequence. + + Status-Line = HTTP-Version SP Status-Code SP Reason-Phrase CRLF + +6.1.1 Status Code and Reason Phrase + + The Status-Code element is a 3-digit integer result code of the + attempt to understand and satisfy the request. These codes are fully + defined in section 10. The Reason-Phrase is intended to give a short + textual description of the Status-Code. The Status-Code is intended + for use by automata and the Reason-Phrase is intended for the human + user. The client is not required to examine or display the Reason- + Phrase. + + + +Fielding, et al. Standards Track [Page 39] + +RFC 2616 HTTP/1.1 June 1999 + + + The first digit of the Status-Code defines the class of response. The + last two digits do not have any categorization role. There are 5 + values for the first digit: + + - 1xx: Informational - Request received, continuing process + + - 2xx: Success - The action was successfully received, + understood, and accepted + + - 3xx: Redirection - Further action must be taken in order to + complete the request + + - 4xx: Client Error - The request contains bad syntax or cannot + be fulfilled + + - 5xx: Server Error - The server failed to fulfill an apparently + valid request + + The individual values of the numeric status codes defined for + HTTP/1.1, and an example set of corresponding Reason-Phrase's, are + presented below. The reason phrases listed here are only + recommendations -- they MAY be replaced by local equivalents without + affecting the protocol. + + Status-Code = + "100" ; Section 10.1.1: Continue + | "101" ; Section 10.1.2: Switching Protocols + | "200" ; Section 10.2.1: OK + | "201" ; Section 10.2.2: Created + | "202" ; Section 10.2.3: Accepted + | "203" ; Section 10.2.4: Non-Authoritative Information + | "204" ; Section 10.2.5: No Content + | "205" ; Section 10.2.6: Reset Content + | "206" ; Section 10.2.7: Partial Content + | "300" ; Section 10.3.1: Multiple Choices + | "301" ; Section 10.3.2: Moved Permanently + | "302" ; Section 10.3.3: Found + | "303" ; Section 10.3.4: See Other + | "304" ; Section 10.3.5: Not Modified + | "305" ; Section 10.3.6: Use Proxy + | "307" ; Section 10.3.8: Temporary Redirect + | "400" ; Section 10.4.1: Bad Request + | "401" ; Section 10.4.2: Unauthorized + | "402" ; Section 10.4.3: Payment Required + | "403" ; Section 10.4.4: Forbidden + | "404" ; Section 10.4.5: Not Found + | "405" ; Section 10.4.6: Method Not Allowed + | "406" ; Section 10.4.7: Not Acceptable + + + +Fielding, et al. Standards Track [Page 40] + +RFC 2616 HTTP/1.1 June 1999 + + + | "407" ; Section 10.4.8: Proxy Authentication Required + | "408" ; Section 10.4.9: Request Time-out + | "409" ; Section 10.4.10: Conflict + | "410" ; Section 10.4.11: Gone + | "411" ; Section 10.4.12: Length Required + | "412" ; Section 10.4.13: Precondition Failed + | "413" ; Section 10.4.14: Request Entity Too Large + | "414" ; Section 10.4.15: Request-URI Too Large + | "415" ; Section 10.4.16: Unsupported Media Type + | "416" ; Section 10.4.17: Requested range not satisfiable + | "417" ; Section 10.4.18: Expectation Failed + | "500" ; Section 10.5.1: Internal Server Error + | "501" ; Section 10.5.2: Not Implemented + | "502" ; Section 10.5.3: Bad Gateway + | "503" ; Section 10.5.4: Service Unavailable + | "504" ; Section 10.5.5: Gateway Time-out + | "505" ; Section 10.5.6: HTTP Version not supported + | extension-code + + extension-code = 3DIGIT + Reason-Phrase = * + + HTTP status codes are extensible. HTTP applications are not required + to understand the meaning of all registered status codes, though such + understanding is obviously desirable. However, applications MUST + understand the class of any status code, as indicated by the first + digit, and treat any unrecognized response as being equivalent to the + x00 status code of that class, with the exception that an + unrecognized response MUST NOT be cached. For example, if an + unrecognized status code of 431 is received by the client, it can + safely assume that there was something wrong with its request and + treat the response as if it had received a 400 status code. In such + cases, user agents SHOULD present to the user the entity returned + with the response, since that entity is likely to include human- + readable information which will explain the unusual status. + +6.2 Response Header Fields + + The response-header fields allow the server to pass additional + information about the response which cannot be placed in the Status- + Line. These header fields give information about the server and about + further access to the resource identified by the Request-URI. + + response-header = Accept-Ranges ; Section 14.5 + | Age ; Section 14.6 + | ETag ; Section 14.19 + | Location ; Section 14.30 + | Proxy-Authenticate ; Section 14.33 + + + +Fielding, et al. Standards Track [Page 41] + +RFC 2616 HTTP/1.1 June 1999 + + + | Retry-After ; Section 14.37 + | Server ; Section 14.38 + | Vary ; Section 14.44 + | WWW-Authenticate ; Section 14.47 + + Response-header field names can be extended reliably only in + combination with a change in the protocol version. However, new or + experimental header fields MAY be given the semantics of response- + header fields if all parties in the communication recognize them to + be response-header fields. Unrecognized header fields are treated as + entity-header fields. + +7 Entity + + Request and Response messages MAY transfer an entity if not otherwise + restricted by the request method or response status code. An entity + consists of entity-header fields and an entity-body, although some + responses will only include the entity-headers. + + In this section, both sender and recipient refer to either the client + or the server, depending on who sends and who receives the entity. + +7.1 Entity Header Fields + + Entity-header fields define metainformation about the entity-body or, + if no body is present, about the resource identified by the request. + Some of this metainformation is OPTIONAL; some might be REQUIRED by + portions of this specification. + + entity-header = Allow ; Section 14.7 + | Content-Encoding ; Section 14.11 + | Content-Language ; Section 14.12 + | Content-Length ; Section 14.13 + | Content-Location ; Section 14.14 + | Content-MD5 ; Section 14.15 + | Content-Range ; Section 14.16 + | Content-Type ; Section 14.17 + | Expires ; Section 14.21 + | Last-Modified ; Section 14.29 + | extension-header + + extension-header = message-header + + The extension-header mechanism allows additional entity-header fields + to be defined without changing the protocol, but these fields cannot + be assumed to be recognizable by the recipient. Unrecognized header + fields SHOULD be ignored by the recipient and MUST be forwarded by + transparent proxies. + + + +Fielding, et al. Standards Track [Page 42] + +RFC 2616 HTTP/1.1 June 1999 + + +7.2 Entity Body + + The entity-body (if any) sent with an HTTP request or response is in + a format and encoding defined by the entity-header fields. + + entity-body = *OCTET + + An entity-body is only present in a message when a message-body is + present, as described in section 4.3. The entity-body is obtained + from the message-body by decoding any Transfer-Encoding that might + have been applied to ensure safe and proper transfer of the message. + +7.2.1 Type + + When an entity-body is included with a message, the data type of that + body is determined via the header fields Content-Type and Content- + Encoding. These define a two-layer, ordered encoding model: + + entity-body := Content-Encoding( Content-Type( data ) ) + + Content-Type specifies the media type of the underlying data. + Content-Encoding may be used to indicate any additional content + codings applied to the data, usually for the purpose of data + compression, that are a property of the requested resource. There is + no default encoding. + + Any HTTP/1.1 message containing an entity-body SHOULD include a + Content-Type header field defining the media type of that body. If + and only if the media type is not given by a Content-Type field, the + recipient MAY attempt to guess the media type via inspection of its + content and/or the name extension(s) of the URI used to identify the + resource. If the media type remains unknown, the recipient SHOULD + treat it as type "application/octet-stream". + +7.2.2 Entity Length + + The entity-length of a message is the length of the message-body + before any transfer-codings have been applied. Section 4.4 defines + how the transfer-length of a message-body is determined. + + + + + + + + + + + + +Fielding, et al. Standards Track [Page 43] + +RFC 2616 HTTP/1.1 June 1999 + + +8 Connections + +8.1 Persistent Connections + +8.1.1 Purpose + + Prior to persistent connections, a separate TCP connection was + established to fetch each URL, increasing the load on HTTP servers + and causing congestion on the Internet. The use of inline images and + other associated data often require a client to make multiple + requests of the same server in a short amount of time. Analysis of + these performance problems and results from a prototype + implementation are available [26] [30]. Implementation experience and + measurements of actual HTTP/1.1 (RFC 2068) implementations show good + results [39]. Alternatives have also been explored, for example, + T/TCP [27]. + + Persistent HTTP connections have a number of advantages: + + - By opening and closing fewer TCP connections, CPU time is saved + in routers and hosts (clients, servers, proxies, gateways, + tunnels, or caches), and memory used for TCP protocol control + blocks can be saved in hosts. + + - HTTP requests and responses can be pipelined on a connection. + Pipelining allows a client to make multiple requests without + waiting for each response, allowing a single TCP connection to + be used much more efficiently, with much lower elapsed time. + + - Network congestion is reduced by reducing the number of packets + caused by TCP opens, and by allowing TCP sufficient time to + determine the congestion state of the network. + + - Latency on subsequent requests is reduced since there is no time + spent in TCP's connection opening handshake. + + - HTTP can evolve more gracefully, since errors can be reported + without the penalty of closing the TCP connection. Clients using + future versions of HTTP might optimistically try a new feature, + but if communicating with an older server, retry with old + semantics after an error is reported. + + HTTP implementations SHOULD implement persistent connections. + + + + + + + + +Fielding, et al. Standards Track [Page 44] + +RFC 2616 HTTP/1.1 June 1999 + + +8.1.2 Overall Operation + + A significant difference between HTTP/1.1 and earlier versions of + HTTP is that persistent connections are the default behavior of any + HTTP connection. That is, unless otherwise indicated, the client + SHOULD assume that the server will maintain a persistent connection, + even after error responses from the server. + + Persistent connections provide a mechanism by which a client and a + server can signal the close of a TCP connection. This signaling takes + place using the Connection header field (section 14.10). Once a close + has been signaled, the client MUST NOT send any more requests on that + connection. + +8.1.2.1 Negotiation + + An HTTP/1.1 server MAY assume that a HTTP/1.1 client intends to + maintain a persistent connection unless a Connection header including + the connection-token "close" was sent in the request. If the server + chooses to close the connection immediately after sending the + response, it SHOULD send a Connection header including the + connection-token close. + + An HTTP/1.1 client MAY expect a connection to remain open, but would + decide to keep it open based on whether the response from a server + contains a Connection header with the connection-token close. In case + the client does not want to maintain a connection for more than that + request, it SHOULD send a Connection header including the + connection-token close. + + If either the client or the server sends the close token in the + Connection header, that request becomes the last one for the + connection. + + Clients and servers SHOULD NOT assume that a persistent connection is + maintained for HTTP versions less than 1.1 unless it is explicitly + signaled. See section 19.6.2 for more information on backward + compatibility with HTTP/1.0 clients. + + In order to remain persistent, all messages on the connection MUST + have a self-defined message length (i.e., one not defined by closure + of the connection), as described in section 4.4. + + + + + + + + + +Fielding, et al. Standards Track [Page 45] + +RFC 2616 HTTP/1.1 June 1999 + + +8.1.2.2 Pipelining + + A client that supports persistent connections MAY "pipeline" its + requests (i.e., send multiple requests without waiting for each + response). A server MUST send its responses to those requests in the + same order that the requests were received. + + Clients which assume persistent connections and pipeline immediately + after connection establishment SHOULD be prepared to retry their + connection if the first pipelined attempt fails. If a client does + such a retry, it MUST NOT pipeline before it knows the connection is + persistent. Clients MUST also be prepared to resend their requests if + the server closes the connection before sending all of the + corresponding responses. + + Clients SHOULD NOT pipeline requests using non-idempotent methods or + non-idempotent sequences of methods (see section 9.1.2). Otherwise, a + premature termination of the transport connection could lead to + indeterminate results. A client wishing to send a non-idempotent + request SHOULD wait to send that request until it has received the + response status for the previous request. + +8.1.3 Proxy Servers + + It is especially important that proxies correctly implement the + properties of the Connection header field as specified in section + 14.10. + + The proxy server MUST signal persistent connections separately with + its clients and the origin servers (or other proxy servers) that it + connects to. Each persistent connection applies to only one transport + link. + + A proxy server MUST NOT establish a HTTP/1.1 persistent connection + with an HTTP/1.0 client (but see RFC 2068 [33] for information and + discussion of the problems with the Keep-Alive header implemented by + many HTTP/1.0 clients). + +8.1.4 Practical Considerations + + Servers will usually have some time-out value beyond which they will + no longer maintain an inactive connection. Proxy servers might make + this a higher value since it is likely that the client will be making + more connections through the same server. The use of persistent + connections places no requirements on the length (or existence) of + this time-out for either the client or the server. + + + + + +Fielding, et al. Standards Track [Page 46] + +RFC 2616 HTTP/1.1 June 1999 + + + When a client or server wishes to time-out it SHOULD issue a graceful + close on the transport connection. Clients and servers SHOULD both + constantly watch for the other side of the transport close, and + respond to it as appropriate. If a client or server does not detect + the other side's close promptly it could cause unnecessary resource + drain on the network. + + A client, server, or proxy MAY close the transport connection at any + time. For example, a client might have started to send a new request + at the same time that the server has decided to close the "idle" + connection. From the server's point of view, the connection is being + closed while it was idle, but from the client's point of view, a + request is in progress. + + This means that clients, servers, and proxies MUST be able to recover + from asynchronous close events. Client software SHOULD reopen the + transport connection and retransmit the aborted sequence of requests + without user interaction so long as the request sequence is + idempotent (see section 9.1.2). Non-idempotent methods or sequences + MUST NOT be automatically retried, although user agents MAY offer a + human operator the choice of retrying the request(s). Confirmation by + user-agent software with semantic understanding of the application + MAY substitute for user confirmation. The automatic retry SHOULD NOT + be repeated if the second sequence of requests fails. + + Servers SHOULD always respond to at least one request per connection, + if at all possible. Servers SHOULD NOT close a connection in the + middle of transmitting a response, unless a network or client failure + is suspected. + + Clients that use persistent connections SHOULD limit the number of + simultaneous connections that they maintain to a given server. A + single-user client SHOULD NOT maintain more than 2 connections with + any server or proxy. A proxy SHOULD use up to 2*N connections to + another server or proxy, where N is the number of simultaneously + active users. These guidelines are intended to improve HTTP response + times and avoid congestion. + +8.2 Message Transmission Requirements + +8.2.1 Persistent Connections and Flow Control + + HTTP/1.1 servers SHOULD maintain persistent connections and use TCP's + flow control mechanisms to resolve temporary overloads, rather than + terminating connections with the expectation that clients will retry. + The latter technique can exacerbate network congestion. + + + + + +Fielding, et al. Standards Track [Page 47] + +RFC 2616 HTTP/1.1 June 1999 + + +8.2.2 Monitoring Connections for Error Status Messages + + An HTTP/1.1 (or later) client sending a message-body SHOULD monitor + the network connection for an error status while it is transmitting + the request. If the client sees an error status, it SHOULD + immediately cease transmitting the body. If the body is being sent + using a "chunked" encoding (section 3.6), a zero length chunk and + empty trailer MAY be used to prematurely mark the end of the message. + If the body was preceded by a Content-Length header, the client MUST + close the connection. + +8.2.3 Use of the 100 (Continue) Status + + The purpose of the 100 (Continue) status (see section 10.1.1) is to + allow a client that is sending a request message with a request body + to determine if the origin server is willing to accept the request + (based on the request headers) before the client sends the request + body. In some cases, it might either be inappropriate or highly + inefficient for the client to send the body if the server will reject + the message without looking at the body. + + Requirements for HTTP/1.1 clients: + + - If a client will wait for a 100 (Continue) response before + sending the request body, it MUST send an Expect request-header + field (section 14.20) with the "100-continue" expectation. + + - A client MUST NOT send an Expect request-header field (section + 14.20) with the "100-continue" expectation if it does not intend + to send a request body. + + Because of the presence of older implementations, the protocol allows + ambiguous situations in which a client may send "Expect: 100- + continue" without receiving either a 417 (Expectation Failed) status + or a 100 (Continue) status. Therefore, when a client sends this + header field to an origin server (possibly via a proxy) from which it + has never seen a 100 (Continue) status, the client SHOULD NOT wait + for an indefinite period before sending the request body. + + Requirements for HTTP/1.1 origin servers: + + - Upon receiving a request which includes an Expect request-header + field with the "100-continue" expectation, an origin server MUST + either respond with 100 (Continue) status and continue to read + from the input stream, or respond with a final status code. The + origin server MUST NOT wait for the request body before sending + the 100 (Continue) response. If it responds with a final status + code, it MAY close the transport connection or it MAY continue + + + +Fielding, et al. Standards Track [Page 48] + +RFC 2616 HTTP/1.1 June 1999 + + + to read and discard the rest of the request. It MUST NOT + perform the requested method if it returns a final status code. + + - An origin server SHOULD NOT send a 100 (Continue) response if + the request message does not include an Expect request-header + field with the "100-continue" expectation, and MUST NOT send a + 100 (Continue) response if such a request comes from an HTTP/1.0 + (or earlier) client. There is an exception to this rule: for + compatibility with RFC 2068, a server MAY send a 100 (Continue) + status in response to an HTTP/1.1 PUT or POST request that does + not include an Expect request-header field with the "100- + continue" expectation. This exception, the purpose of which is + to minimize any client processing delays associated with an + undeclared wait for 100 (Continue) status, applies only to + HTTP/1.1 requests, and not to requests with any other HTTP- + version value. + + - An origin server MAY omit a 100 (Continue) response if it has + already received some or all of the request body for the + corresponding request. + + - An origin server that sends a 100 (Continue) response MUST + ultimately send a final status code, once the request body is + received and processed, unless it terminates the transport + connection prematurely. + + - If an origin server receives a request that does not include an + Expect request-header field with the "100-continue" expectation, + the request includes a request body, and the server responds + with a final status code before reading the entire request body + from the transport connection, then the server SHOULD NOT close + the transport connection until it has read the entire request, + or until the client closes the connection. Otherwise, the client + might not reliably receive the response message. However, this + requirement is not be construed as preventing a server from + defending itself against denial-of-service attacks, or from + badly broken client implementations. + + Requirements for HTTP/1.1 proxies: + + - If a proxy receives a request that includes an Expect request- + header field with the "100-continue" expectation, and the proxy + either knows that the next-hop server complies with HTTP/1.1 or + higher, or does not know the HTTP version of the next-hop + server, it MUST forward the request, including the Expect header + field. + + + + + +Fielding, et al. Standards Track [Page 49] + +RFC 2616 HTTP/1.1 June 1999 + + + - If the proxy knows that the version of the next-hop server is + HTTP/1.0 or lower, it MUST NOT forward the request, and it MUST + respond with a 417 (Expectation Failed) status. + + - Proxies SHOULD maintain a cache recording the HTTP version + numbers received from recently-referenced next-hop servers. + + - A proxy MUST NOT forward a 100 (Continue) response if the + request message was received from an HTTP/1.0 (or earlier) + client and did not include an Expect request-header field with + the "100-continue" expectation. This requirement overrides the + general rule for forwarding of 1xx responses (see section 10.1). + +8.2.4 Client Behavior if Server Prematurely Closes Connection + + If an HTTP/1.1 client sends a request which includes a request body, + but which does not include an Expect request-header field with the + "100-continue" expectation, and if the client is not directly + connected to an HTTP/1.1 origin server, and if the client sees the + connection close before receiving any status from the server, the + client SHOULD retry the request. If the client does retry this + request, it MAY use the following "binary exponential backoff" + algorithm to be assured of obtaining a reliable response: + + 1. Initiate a new connection to the server + + 2. Transmit the request-headers + + 3. Initialize a variable R to the estimated round-trip time to the + server (e.g., based on the time it took to establish the + connection), or to a constant value of 5 seconds if the round- + trip time is not available. + + 4. Compute T = R * (2**N), where N is the number of previous + retries of this request. + + 5. Wait either for an error response from the server, or for T + seconds (whichever comes first) + + 6. If no error response is received, after T seconds transmit the + body of the request. + + 7. If client sees that the connection is closed prematurely, + repeat from step 1 until the request is accepted, an error + response is received, or the user becomes impatient and + terminates the retry process. + + + + + +Fielding, et al. Standards Track [Page 50] + +RFC 2616 HTTP/1.1 June 1999 + + + If at any point an error status is received, the client + + - SHOULD NOT continue and + + - SHOULD close the connection if it has not completed sending the + request message. + +9 Method Definitions + + The set of common methods for HTTP/1.1 is defined below. Although + this set can be expanded, additional methods cannot be assumed to + share the same semantics for separately extended clients and servers. + + The Host request-header field (section 14.23) MUST accompany all + HTTP/1.1 requests. + +9.1 Safe and Idempotent Methods + +9.1.1 Safe Methods + + Implementors should be aware that the software represents the user in + their interactions over the Internet, and should be careful to allow + the user to be aware of any actions they might take which may have an + unexpected significance to themselves or others. + + In particular, the convention has been established that the GET and + HEAD methods SHOULD NOT have the significance of taking an action + other than retrieval. These methods ought to be considered "safe". + This allows user agents to represent other methods, such as POST, PUT + and DELETE, in a special way, so that the user is made aware of the + fact that a possibly unsafe action is being requested. + + Naturally, it is not possible to ensure that the server does not + generate side-effects as a result of performing a GET request; in + fact, some dynamic resources consider that a feature. The important + distinction here is that the user did not request the side-effects, + so therefore cannot be held accountable for them. + +9.1.2 Idempotent Methods + + Methods can also have the property of "idempotence" in that (aside + from error or expiration issues) the side-effects of N > 0 identical + requests is the same as for a single request. The methods GET, HEAD, + PUT and DELETE share this property. Also, the methods OPTIONS and + TRACE SHOULD NOT have side effects, and so are inherently idempotent. + + + + + + +Fielding, et al. Standards Track [Page 51] + +RFC 2616 HTTP/1.1 June 1999 + + + However, it is possible that a sequence of several requests is non- + idempotent, even if all of the methods executed in that sequence are + idempotent. (A sequence is idempotent if a single execution of the + entire sequence always yields a result that is not changed by a + reexecution of all, or part, of that sequence.) For example, a + sequence is non-idempotent if its result depends on a value that is + later modified in the same sequence. + + A sequence that never has side effects is idempotent, by definition + (provided that no concurrent operations are being executed on the + same set of resources). + +9.2 OPTIONS + + The OPTIONS method represents a request for information about the + communication options available on the request/response chain + identified by the Request-URI. This method allows the client to + determine the options and/or requirements associated with a resource, + or the capabilities of a server, without implying a resource action + or initiating a resource retrieval. + + Responses to this method are not cacheable. + + If the OPTIONS request includes an entity-body (as indicated by the + presence of Content-Length or Transfer-Encoding), then the media type + MUST be indicated by a Content-Type field. Although this + specification does not define any use for such a body, future + extensions to HTTP might use the OPTIONS body to make more detailed + queries on the server. A server that does not support such an + extension MAY discard the request body. + + If the Request-URI is an asterisk ("*"), the OPTIONS request is + intended to apply to the server in general rather than to a specific + resource. Since a server's communication options typically depend on + the resource, the "*" request is only useful as a "ping" or "no-op" + type of method; it does nothing beyond allowing the client to test + the capabilities of the server. For example, this can be used to test + a proxy for HTTP/1.1 compliance (or lack thereof). + + If the Request-URI is not an asterisk, the OPTIONS request applies + only to the options that are available when communicating with that + resource. + + A 200 response SHOULD include any header fields that indicate + optional features implemented by the server and applicable to that + resource (e.g., Allow), possibly including extensions not defined by + this specification. The response body, if any, SHOULD also include + information about the communication options. The format for such a + + + +Fielding, et al. Standards Track [Page 52] + +RFC 2616 HTTP/1.1 June 1999 + + + body is not defined by this specification, but might be defined by + future extensions to HTTP. Content negotiation MAY be used to select + the appropriate response format. If no response body is included, the + response MUST include a Content-Length field with a field-value of + "0". + + The Max-Forwards request-header field MAY be used to target a + specific proxy in the request chain. When a proxy receives an OPTIONS + request on an absoluteURI for which request forwarding is permitted, + the proxy MUST check for a Max-Forwards field. If the Max-Forwards + field-value is zero ("0"), the proxy MUST NOT forward the message; + instead, the proxy SHOULD respond with its own communication options. + If the Max-Forwards field-value is an integer greater than zero, the + proxy MUST decrement the field-value when it forwards the request. If + no Max-Forwards field is present in the request, then the forwarded + request MUST NOT include a Max-Forwards field. + +9.3 GET + + The GET method means retrieve whatever information (in the form of an + entity) is identified by the Request-URI. If the Request-URI refers + to a data-producing process, it is the produced data which shall be + returned as the entity in the response and not the source text of the + process, unless that text happens to be the output of the process. + + The semantics of the GET method change to a "conditional GET" if the + request message includes an If-Modified-Since, If-Unmodified-Since, + If-Match, If-None-Match, or If-Range header field. A conditional GET + method requests that the entity be transferred only under the + circumstances described by the conditional header field(s). The + conditional GET method is intended to reduce unnecessary network + usage by allowing cached entities to be refreshed without requiring + multiple requests or transferring data already held by the client. + + The semantics of the GET method change to a "partial GET" if the + request message includes a Range header field. A partial GET requests + that only part of the entity be transferred, as described in section + 14.35. The partial GET method is intended to reduce unnecessary + network usage by allowing partially-retrieved entities to be + completed without transferring data already held by the client. + + The response to a GET request is cacheable if and only if it meets + the requirements for HTTP caching described in section 13. + + See section 15.1.3 for security considerations when used for forms. + + + + + + +Fielding, et al. Standards Track [Page 53] + +RFC 2616 HTTP/1.1 June 1999 + + +9.4 HEAD + + The HEAD method is identical to GET except that the server MUST NOT + return a message-body in the response. The metainformation contained + in the HTTP headers in response to a HEAD request SHOULD be identical + to the information sent in response to a GET request. This method can + be used for obtaining metainformation about the entity implied by the + request without transferring the entity-body itself. This method is + often used for testing hypertext links for validity, accessibility, + and recent modification. + + The response to a HEAD request MAY be cacheable in the sense that the + information contained in the response MAY be used to update a + previously cached entity from that resource. If the new field values + indicate that the cached entity differs from the current entity (as + would be indicated by a change in Content-Length, Content-MD5, ETag + or Last-Modified), then the cache MUST treat the cache entry as + stale. + +9.5 POST + + The POST method is used to request that the origin server accept the + entity enclosed in the request as a new subordinate of the resource + identified by the Request-URI in the Request-Line. POST is designed + to allow a uniform method to cover the following functions: + + - Annotation of existing resources; + + - Posting a message to a bulletin board, newsgroup, mailing list, + or similar group of articles; + + - Providing a block of data, such as the result of submitting a + form, to a data-handling process; + + - Extending a database through an append operation. + + The actual function performed by the POST method is determined by the + server and is usually dependent on the Request-URI. The posted entity + is subordinate to that URI in the same way that a file is subordinate + to a directory containing it, a news article is subordinate to a + newsgroup to which it is posted, or a record is subordinate to a + database. + + The action performed by the POST method might not result in a + resource that can be identified by a URI. In this case, either 200 + (OK) or 204 (No Content) is the appropriate response status, + depending on whether or not the response includes an entity that + describes the result. + + + +Fielding, et al. Standards Track [Page 54] + +RFC 2616 HTTP/1.1 June 1999 + + + If a resource has been created on the origin server, the response + SHOULD be 201 (Created) and contain an entity which describes the + status of the request and refers to the new resource, and a Location + header (see section 14.30). + + Responses to this method are not cacheable, unless the response + includes appropriate Cache-Control or Expires header fields. However, + the 303 (See Other) response can be used to direct the user agent to + retrieve a cacheable resource. + + POST requests MUST obey the message transmission requirements set out + in section 8.2. + + See section 15.1.3 for security considerations. + +9.6 PUT + + The PUT method requests that the enclosed entity be stored under the + supplied Request-URI. If the Request-URI refers to an already + existing resource, the enclosed entity SHOULD be considered as a + modified version of the one residing on the origin server. If the + Request-URI does not point to an existing resource, and that URI is + capable of being defined as a new resource by the requesting user + agent, the origin server can create the resource with that URI. If a + new resource is created, the origin server MUST inform the user agent + via the 201 (Created) response. If an existing resource is modified, + either the 200 (OK) or 204 (No Content) response codes SHOULD be sent + to indicate successful completion of the request. If the resource + could not be created or modified with the Request-URI, an appropriate + error response SHOULD be given that reflects the nature of the + problem. The recipient of the entity MUST NOT ignore any Content-* + (e.g. Content-Range) headers that it does not understand or implement + and MUST return a 501 (Not Implemented) response in such cases. + + If the request passes through a cache and the Request-URI identifies + one or more currently cached entities, those entries SHOULD be + treated as stale. Responses to this method are not cacheable. + + The fundamental difference between the POST and PUT requests is + reflected in the different meaning of the Request-URI. The URI in a + POST request identifies the resource that will handle the enclosed + entity. That resource might be a data-accepting process, a gateway to + some other protocol, or a separate entity that accepts annotations. + In contrast, the URI in a PUT request identifies the entity enclosed + with the request -- the user agent knows what URI is intended and the + server MUST NOT attempt to apply the request to some other resource. + If the server desires that the request be applied to a different URI, + + + + +Fielding, et al. Standards Track [Page 55] + +RFC 2616 HTTP/1.1 June 1999 + + + it MUST send a 301 (Moved Permanently) response; the user agent MAY + then make its own decision regarding whether or not to redirect the + request. + + A single resource MAY be identified by many different URIs. For + example, an article might have a URI for identifying "the current + version" which is separate from the URI identifying each particular + version. In this case, a PUT request on a general URI might result in + several other URIs being defined by the origin server. + + HTTP/1.1 does not define how a PUT method affects the state of an + origin server. + + PUT requests MUST obey the message transmission requirements set out + in section 8.2. + + Unless otherwise specified for a particular entity-header, the + entity-headers in the PUT request SHOULD be applied to the resource + created or modified by the PUT. + +9.7 DELETE + + The DELETE method requests that the origin server delete the resource + identified by the Request-URI. This method MAY be overridden by human + intervention (or other means) on the origin server. The client cannot + be guaranteed that the operation has been carried out, even if the + status code returned from the origin server indicates that the action + has been completed successfully. However, the server SHOULD NOT + indicate success unless, at the time the response is given, it + intends to delete the resource or move it to an inaccessible + location. + + A successful response SHOULD be 200 (OK) if the response includes an + entity describing the status, 202 (Accepted) if the action has not + yet been enacted, or 204 (No Content) if the action has been enacted + but the response does not include an entity. + + If the request passes through a cache and the Request-URI identifies + one or more currently cached entities, those entries SHOULD be + treated as stale. Responses to this method are not cacheable. + +9.8 TRACE + + The TRACE method is used to invoke a remote, application-layer loop- + back of the request message. The final recipient of the request + SHOULD reflect the message received back to the client as the + entity-body of a 200 (OK) response. The final recipient is either the + + + + +Fielding, et al. Standards Track [Page 56] + +RFC 2616 HTTP/1.1 June 1999 + + + origin server or the first proxy or gateway to receive a Max-Forwards + value of zero (0) in the request (see section 14.31). A TRACE request + MUST NOT include an entity. + + TRACE allows the client to see what is being received at the other + end of the request chain and use that data for testing or diagnostic + information. The value of the Via header field (section 14.45) is of + particular interest, since it acts as a trace of the request chain. + Use of the Max-Forwards header field allows the client to limit the + length of the request chain, which is useful for testing a chain of + proxies forwarding messages in an infinite loop. + + If the request is valid, the response SHOULD contain the entire + request message in the entity-body, with a Content-Type of + "message/http". Responses to this method MUST NOT be cached. + +9.9 CONNECT + + This specification reserves the method name CONNECT for use with a + proxy that can dynamically switch to being a tunnel (e.g. SSL + tunneling [44]). + +10 Status Code Definitions + + Each Status-Code is described below, including a description of which + method(s) it can follow and any metainformation required in the + response. + +10.1 Informational 1xx + + This class of status code indicates a provisional response, + consisting only of the Status-Line and optional headers, and is + terminated by an empty line. There are no required headers for this + class of status code. Since HTTP/1.0 did not define any 1xx status + codes, servers MUST NOT send a 1xx response to an HTTP/1.0 client + except under experimental conditions. + + A client MUST be prepared to accept one or more 1xx status responses + prior to a regular response, even if the client does not expect a 100 + (Continue) status message. Unexpected 1xx status responses MAY be + ignored by a user agent. + + Proxies MUST forward 1xx responses, unless the connection between the + proxy and its client has been closed, or unless the proxy itself + requested the generation of the 1xx response. (For example, if a + + + + + + +Fielding, et al. Standards Track [Page 57] + +RFC 2616 HTTP/1.1 June 1999 + + + proxy adds a "Expect: 100-continue" field when it forwards a request, + then it need not forward the corresponding 100 (Continue) + response(s).) + +10.1.1 100 Continue + + The client SHOULD continue with its request. This interim response is + used to inform the client that the initial part of the request has + been received and has not yet been rejected by the server. The client + SHOULD continue by sending the remainder of the request or, if the + request has already been completed, ignore this response. The server + MUST send a final response after the request has been completed. See + section 8.2.3 for detailed discussion of the use and handling of this + status code. + +10.1.2 101 Switching Protocols + + The server understands and is willing to comply with the client's + request, via the Upgrade message header field (section 14.42), for a + change in the application protocol being used on this connection. The + server will switch protocols to those defined by the response's + Upgrade header field immediately after the empty line which + terminates the 101 response. + + The protocol SHOULD be switched only when it is advantageous to do + so. For example, switching to a newer version of HTTP is advantageous + over older versions, and switching to a real-time, synchronous + protocol might be advantageous when delivering resources that use + such features. + +10.2 Successful 2xx + + This class of status code indicates that the client's request was + successfully received, understood, and accepted. + +10.2.1 200 OK + + The request has succeeded. The information returned with the response + is dependent on the method used in the request, for example: + + GET an entity corresponding to the requested resource is sent in + the response; + + HEAD the entity-header fields corresponding to the requested + resource are sent in the response without any message-body; + + POST an entity describing or containing the result of the action; + + + + +Fielding, et al. Standards Track [Page 58] + +RFC 2616 HTTP/1.1 June 1999 + + + TRACE an entity containing the request message as received by the + end server. + +10.2.2 201 Created + + The request has been fulfilled and resulted in a new resource being + created. The newly created resource can be referenced by the URI(s) + returned in the entity of the response, with the most specific URI + for the resource given by a Location header field. The response + SHOULD include an entity containing a list of resource + characteristics and location(s) from which the user or user agent can + choose the one most appropriate. The entity format is specified by + the media type given in the Content-Type header field. The origin + server MUST create the resource before returning the 201 status code. + If the action cannot be carried out immediately, the server SHOULD + respond with 202 (Accepted) response instead. + + A 201 response MAY contain an ETag response header field indicating + the current value of the entity tag for the requested variant just + created, see section 14.19. + +10.2.3 202 Accepted + + The request has been accepted for processing, but the processing has + not been completed. The request might or might not eventually be + acted upon, as it might be disallowed when processing actually takes + place. There is no facility for re-sending a status code from an + asynchronous operation such as this. + + The 202 response is intentionally non-committal. Its purpose is to + allow a server to accept a request for some other process (perhaps a + batch-oriented process that is only run once per day) without + requiring that the user agent's connection to the server persist + until the process is completed. The entity returned with this + response SHOULD include an indication of the request's current status + and either a pointer to a status monitor or some estimate of when the + user can expect the request to be fulfilled. + +10.2.4 203 Non-Authoritative Information + + The returned metainformation in the entity-header is not the + definitive set as available from the origin server, but is gathered + from a local or a third-party copy. The set presented MAY be a subset + or superset of the original version. For example, including local + annotation information about the resource might result in a superset + of the metainformation known by the origin server. Use of this + response code is not required and is only appropriate when the + response would otherwise be 200 (OK). + + + +Fielding, et al. Standards Track [Page 59] + +RFC 2616 HTTP/1.1 June 1999 + + +10.2.5 204 No Content + + The server has fulfilled the request but does not need to return an + entity-body, and might want to return updated metainformation. The + response MAY include new or updated metainformation in the form of + entity-headers, which if present SHOULD be associated with the + requested variant. + + If the client is a user agent, it SHOULD NOT change its document view + from that which caused the request to be sent. This response is + primarily intended to allow input for actions to take place without + causing a change to the user agent's active document view, although + any new or updated metainformation SHOULD be applied to the document + currently in the user agent's active view. + + The 204 response MUST NOT include a message-body, and thus is always + terminated by the first empty line after the header fields. + +10.2.6 205 Reset Content + + The server has fulfilled the request and the user agent SHOULD reset + the document view which caused the request to be sent. This response + is primarily intended to allow input for actions to take place via + user input, followed by a clearing of the form in which the input is + given so that the user can easily initiate another input action. The + response MUST NOT include an entity. + +10.2.7 206 Partial Content + + The server has fulfilled the partial GET request for the resource. + The request MUST have included a Range header field (section 14.35) + indicating the desired range, and MAY have included an If-Range + header field (section 14.27) to make the request conditional. + + The response MUST include the following header fields: + + - Either a Content-Range header field (section 14.16) indicating + the range included with this response, or a multipart/byteranges + Content-Type including Content-Range fields for each part. If a + Content-Length header field is present in the response, its + value MUST match the actual number of OCTETs transmitted in the + message-body. + + - Date + + - ETag and/or Content-Location, if the header would have been sent + in a 200 response to the same request + + + + +Fielding, et al. Standards Track [Page 60] + +RFC 2616 HTTP/1.1 June 1999 + + + - Expires, Cache-Control, and/or Vary, if the field-value might + differ from that sent in any previous response for the same + variant + + If the 206 response is the result of an If-Range request that used a + strong cache validator (see section 13.3.3), the response SHOULD NOT + include other entity-headers. If the response is the result of an + If-Range request that used a weak validator, the response MUST NOT + include other entity-headers; this prevents inconsistencies between + cached entity-bodies and updated headers. Otherwise, the response + MUST include all of the entity-headers that would have been returned + with a 200 (OK) response to the same request. + + A cache MUST NOT combine a 206 response with other previously cached + content if the ETag or Last-Modified headers do not match exactly, + see 13.5.4. + + A cache that does not support the Range and Content-Range headers + MUST NOT cache 206 (Partial) responses. + +10.3 Redirection 3xx + + This class of status code indicates that further action needs to be + taken by the user agent in order to fulfill the request. The action + required MAY be carried out by the user agent without interaction + with the user if and only if the method used in the second request is + GET or HEAD. A client SHOULD detect infinite redirection loops, since + such loops generate network traffic for each redirection. + + Note: previous versions of this specification recommended a + maximum of five redirections. Content developers should be aware + that there might be clients that implement such a fixed + limitation. + +10.3.1 300 Multiple Choices + + The requested resource corresponds to any one of a set of + representations, each with its own specific location, and agent- + driven negotiation information (section 12) is being provided so that + the user (or user agent) can select a preferred representation and + redirect its request to that location. + + Unless it was a HEAD request, the response SHOULD include an entity + containing a list of resource characteristics and location(s) from + which the user or user agent can choose the one most appropriate. The + entity format is specified by the media type given in the Content- + Type header field. Depending upon the format and the capabilities of + + + + +Fielding, et al. Standards Track [Page 61] + +RFC 2616 HTTP/1.1 June 1999 + + + the user agent, selection of the most appropriate choice MAY be + performed automatically. However, this specification does not define + any standard for such automatic selection. + + If the server has a preferred choice of representation, it SHOULD + include the specific URI for that representation in the Location + field; user agents MAY use the Location field value for automatic + redirection. This response is cacheable unless indicated otherwise. + +10.3.2 301 Moved Permanently + + The requested resource has been assigned a new permanent URI and any + future references to this resource SHOULD use one of the returned + URIs. Clients with link editing capabilities ought to automatically + re-link references to the Request-URI to one or more of the new + references returned by the server, where possible. This response is + cacheable unless indicated otherwise. + + The new permanent URI SHOULD be given by the Location field in the + response. Unless the request method was HEAD, the entity of the + response SHOULD contain a short hypertext note with a hyperlink to + the new URI(s). + + If the 301 status code is received in response to a request other + than GET or HEAD, the user agent MUST NOT automatically redirect the + request unless it can be confirmed by the user, since this might + change the conditions under which the request was issued. + + Note: When automatically redirecting a POST request after + receiving a 301 status code, some existing HTTP/1.0 user agents + will erroneously change it into a GET request. + +10.3.3 302 Found + + The requested resource resides temporarily under a different URI. + Since the redirection might be altered on occasion, the client SHOULD + continue to use the Request-URI for future requests. This response + is only cacheable if indicated by a Cache-Control or Expires header + field. + + The temporary URI SHOULD be given by the Location field in the + response. Unless the request method was HEAD, the entity of the + response SHOULD contain a short hypertext note with a hyperlink to + the new URI(s). + + + + + + + +Fielding, et al. Standards Track [Page 62] + +RFC 2616 HTTP/1.1 June 1999 + + + If the 302 status code is received in response to a request other + than GET or HEAD, the user agent MUST NOT automatically redirect the + request unless it can be confirmed by the user, since this might + change the conditions under which the request was issued. + + Note: RFC 1945 and RFC 2068 specify that the client is not allowed + to change the method on the redirected request. However, most + existing user agent implementations treat 302 as if it were a 303 + response, performing a GET on the Location field-value regardless + of the original request method. The status codes 303 and 307 have + been added for servers that wish to make unambiguously clear which + kind of reaction is expected of the client. + +10.3.4 303 See Other + + The response to the request can be found under a different URI and + SHOULD be retrieved using a GET method on that resource. This method + exists primarily to allow the output of a POST-activated script to + redirect the user agent to a selected resource. The new URI is not a + substitute reference for the originally requested resource. The 303 + response MUST NOT be cached, but the response to the second + (redirected) request might be cacheable. + + The different URI SHOULD be given by the Location field in the + response. Unless the request method was HEAD, the entity of the + response SHOULD contain a short hypertext note with a hyperlink to + the new URI(s). + + Note: Many pre-HTTP/1.1 user agents do not understand the 303 + status. When interoperability with such clients is a concern, the + 302 status code may be used instead, since most user agents react + to a 302 response as described here for 303. + +10.3.5 304 Not Modified + + If the client has performed a conditional GET request and access is + allowed, but the document has not been modified, the server SHOULD + respond with this status code. The 304 response MUST NOT contain a + message-body, and thus is always terminated by the first empty line + after the header fields. + + The response MUST include the following header fields: + + - Date, unless its omission is required by section 14.18.1 + + + + + + + +Fielding, et al. Standards Track [Page 63] + +RFC 2616 HTTP/1.1 June 1999 + + + If a clockless origin server obeys these rules, and proxies and + clients add their own Date to any response received without one (as + already specified by [RFC 2068], section 14.19), caches will operate + correctly. + + - ETag and/or Content-Location, if the header would have been sent + in a 200 response to the same request + + - Expires, Cache-Control, and/or Vary, if the field-value might + differ from that sent in any previous response for the same + variant + + If the conditional GET used a strong cache validator (see section + 13.3.3), the response SHOULD NOT include other entity-headers. + Otherwise (i.e., the conditional GET used a weak validator), the + response MUST NOT include other entity-headers; this prevents + inconsistencies between cached entity-bodies and updated headers. + + If a 304 response indicates an entity not currently cached, then the + cache MUST disregard the response and repeat the request without the + conditional. + + If a cache uses a received 304 response to update a cache entry, the + cache MUST update the entry to reflect any new field values given in + the response. + +10.3.6 305 Use Proxy + + The requested resource MUST be accessed through the proxy given by + the Location field. The Location field gives the URI of the proxy. + The recipient is expected to repeat this single request via the + proxy. 305 responses MUST only be generated by origin servers. + + Note: RFC 2068 was not clear that 305 was intended to redirect a + single request, and to be generated by origin servers only. Not + observing these limitations has significant security consequences. + +10.3.7 306 (Unused) + + The 306 status code was used in a previous version of the + specification, is no longer used, and the code is reserved. + + + + + + + + + + +Fielding, et al. Standards Track [Page 64] + +RFC 2616 HTTP/1.1 June 1999 + + +10.3.8 307 Temporary Redirect + + The requested resource resides temporarily under a different URI. + Since the redirection MAY be altered on occasion, the client SHOULD + continue to use the Request-URI for future requests. This response + is only cacheable if indicated by a Cache-Control or Expires header + field. + + The temporary URI SHOULD be given by the Location field in the + response. Unless the request method was HEAD, the entity of the + response SHOULD contain a short hypertext note with a hyperlink to + the new URI(s) , since many pre-HTTP/1.1 user agents do not + understand the 307 status. Therefore, the note SHOULD contain the + information necessary for a user to repeat the original request on + the new URI. + + If the 307 status code is received in response to a request other + than GET or HEAD, the user agent MUST NOT automatically redirect the + request unless it can be confirmed by the user, since this might + change the conditions under which the request was issued. + +10.4 Client Error 4xx + + The 4xx class of status code is intended for cases in which the + client seems to have erred. Except when responding to a HEAD request, + the server SHOULD include an entity containing an explanation of the + error situation, and whether it is a temporary or permanent + condition. These status codes are applicable to any request method. + User agents SHOULD display any included entity to the user. + + If the client is sending data, a server implementation using TCP + SHOULD be careful to ensure that the client acknowledges receipt of + the packet(s) containing the response, before the server closes the + input connection. If the client continues sending data to the server + after the close, the server's TCP stack will send a reset packet to + the client, which may erase the client's unacknowledged input buffers + before they can be read and interpreted by the HTTP application. + +10.4.1 400 Bad Request + + The request could not be understood by the server due to malformed + syntax. The client SHOULD NOT repeat the request without + modifications. + + + + + + + + +Fielding, et al. Standards Track [Page 65] + +RFC 2616 HTTP/1.1 June 1999 + + +10.4.2 401 Unauthorized + + The request requires user authentication. The response MUST include a + WWW-Authenticate header field (section 14.47) containing a challenge + applicable to the requested resource. The client MAY repeat the + request with a suitable Authorization header field (section 14.8). If + the request already included Authorization credentials, then the 401 + response indicates that authorization has been refused for those + credentials. If the 401 response contains the same challenge as the + prior response, and the user agent has already attempted + authentication at least once, then the user SHOULD be presented the + entity that was given in the response, since that entity might + include relevant diagnostic information. HTTP access authentication + is explained in "HTTP Authentication: Basic and Digest Access + Authentication" [43]. + +10.4.3 402 Payment Required + + This code is reserved for future use. + +10.4.4 403 Forbidden + + The server understood the request, but is refusing to fulfill it. + Authorization will not help and the request SHOULD NOT be repeated. + If the request method was not HEAD and the server wishes to make + public why the request has not been fulfilled, it SHOULD describe the + reason for the refusal in the entity. If the server does not wish to + make this information available to the client, the status code 404 + (Not Found) can be used instead. + +10.4.5 404 Not Found + + The server has not found anything matching the Request-URI. No + indication is given of whether the condition is temporary or + permanent. The 410 (Gone) status code SHOULD be used if the server + knows, through some internally configurable mechanism, that an old + resource is permanently unavailable and has no forwarding address. + This status code is commonly used when the server does not wish to + reveal exactly why the request has been refused, or when no other + response is applicable. + +10.4.6 405 Method Not Allowed + + The method specified in the Request-Line is not allowed for the + resource identified by the Request-URI. The response MUST include an + Allow header containing a list of valid methods for the requested + resource. + + + + +Fielding, et al. Standards Track [Page 66] + +RFC 2616 HTTP/1.1 June 1999 + + +10.4.7 406 Not Acceptable + + The resource identified by the request is only capable of generating + response entities which have content characteristics not acceptable + according to the accept headers sent in the request. + + Unless it was a HEAD request, the response SHOULD include an entity + containing a list of available entity characteristics and location(s) + from which the user or user agent can choose the one most + appropriate. The entity format is specified by the media type given + in the Content-Type header field. Depending upon the format and the + capabilities of the user agent, selection of the most appropriate + choice MAY be performed automatically. However, this specification + does not define any standard for such automatic selection. + + Note: HTTP/1.1 servers are allowed to return responses which are + not acceptable according to the accept headers sent in the + request. In some cases, this may even be preferable to sending a + 406 response. User agents are encouraged to inspect the headers of + an incoming response to determine if it is acceptable. + + If the response could be unacceptable, a user agent SHOULD + temporarily stop receipt of more data and query the user for a + decision on further actions. + +10.4.8 407 Proxy Authentication Required + + This code is similar to 401 (Unauthorized), but indicates that the + client must first authenticate itself with the proxy. The proxy MUST + return a Proxy-Authenticate header field (section 14.33) containing a + challenge applicable to the proxy for the requested resource. The + client MAY repeat the request with a suitable Proxy-Authorization + header field (section 14.34). HTTP access authentication is explained + in "HTTP Authentication: Basic and Digest Access Authentication" + [43]. + +10.4.9 408 Request Timeout + + The client did not produce a request within the time that the server + was prepared to wait. The client MAY repeat the request without + modifications at any later time. + +10.4.10 409 Conflict + + The request could not be completed due to a conflict with the current + state of the resource. This code is only allowed in situations where + it is expected that the user might be able to resolve the conflict + and resubmit the request. The response body SHOULD include enough + + + +Fielding, et al. Standards Track [Page 67] + +RFC 2616 HTTP/1.1 June 1999 + + + information for the user to recognize the source of the conflict. + Ideally, the response entity would include enough information for the + user or user agent to fix the problem; however, that might not be + possible and is not required. + + Conflicts are most likely to occur in response to a PUT request. For + example, if versioning were being used and the entity being PUT + included changes to a resource which conflict with those made by an + earlier (third-party) request, the server might use the 409 response + to indicate that it can't complete the request. In this case, the + response entity would likely contain a list of the differences + between the two versions in a format defined by the response + Content-Type. + +10.4.11 410 Gone + + The requested resource is no longer available at the server and no + forwarding address is known. This condition is expected to be + considered permanent. Clients with link editing capabilities SHOULD + delete references to the Request-URI after user approval. If the + server does not know, or has no facility to determine, whether or not + the condition is permanent, the status code 404 (Not Found) SHOULD be + used instead. This response is cacheable unless indicated otherwise. + + The 410 response is primarily intended to assist the task of web + maintenance by notifying the recipient that the resource is + intentionally unavailable and that the server owners desire that + remote links to that resource be removed. Such an event is common for + limited-time, promotional services and for resources belonging to + individuals no longer working at the server's site. It is not + necessary to mark all permanently unavailable resources as "gone" or + to keep the mark for any length of time -- that is left to the + discretion of the server owner. + +10.4.12 411 Length Required + + The server refuses to accept the request without a defined Content- + Length. The client MAY repeat the request if it adds a valid + Content-Length header field containing the length of the message-body + in the request message. + +10.4.13 412 Precondition Failed + + The precondition given in one or more of the request-header fields + evaluated to false when it was tested on the server. This response + code allows the client to place preconditions on the current resource + metainformation (header field data) and thus prevent the requested + method from being applied to a resource other than the one intended. + + + +Fielding, et al. Standards Track [Page 68] + +RFC 2616 HTTP/1.1 June 1999 + + +10.4.14 413 Request Entity Too Large + + The server is refusing to process a request because the request + entity is larger than the server is willing or able to process. The + server MAY close the connection to prevent the client from continuing + the request. + + If the condition is temporary, the server SHOULD include a Retry- + After header field to indicate that it is temporary and after what + time the client MAY try again. + +10.4.15 414 Request-URI Too Long + + The server is refusing to service the request because the Request-URI + is longer than the server is willing to interpret. This rare + condition is only likely to occur when a client has improperly + converted a POST request to a GET request with long query + information, when the client has descended into a URI "black hole" of + redirection (e.g., a redirected URI prefix that points to a suffix of + itself), or when the server is under attack by a client attempting to + exploit security holes present in some servers using fixed-length + buffers for reading or manipulating the Request-URI. + +10.4.16 415 Unsupported Media Type + + The server is refusing to service the request because the entity of + the request is in a format not supported by the requested resource + for the requested method. + +10.4.17 416 Requested Range Not Satisfiable + + A server SHOULD return a response with this status code if a request + included a Range request-header field (section 14.35), and none of + the range-specifier values in this field overlap the current extent + of the selected resource, and the request did not include an If-Range + request-header field. (For byte-ranges, this means that the first- + byte-pos of all of the byte-range-spec values were greater than the + current length of the selected resource.) + + When this status code is returned for a byte-range request, the + response SHOULD include a Content-Range entity-header field + specifying the current length of the selected resource (see section + 14.16). This response MUST NOT use the multipart/byteranges content- + type. + + + + + + + +Fielding, et al. Standards Track [Page 69] + +RFC 2616 HTTP/1.1 June 1999 + + +10.4.18 417 Expectation Failed + + The expectation given in an Expect request-header field (see section + 14.20) could not be met by this server, or, if the server is a proxy, + the server has unambiguous evidence that the request could not be met + by the next-hop server. + +10.5 Server Error 5xx + + Response status codes beginning with the digit "5" indicate cases in + which the server is aware that it has erred or is incapable of + performing the request. Except when responding to a HEAD request, the + server SHOULD include an entity containing an explanation of the + error situation, and whether it is a temporary or permanent + condition. User agents SHOULD display any included entity to the + user. These response codes are applicable to any request method. + +10.5.1 500 Internal Server Error + + The server encountered an unexpected condition which prevented it + from fulfilling the request. + +10.5.2 501 Not Implemented + + The server does not support the functionality required to fulfill the + request. This is the appropriate response when the server does not + recognize the request method and is not capable of supporting it for + any resource. + +10.5.3 502 Bad Gateway + + The server, while acting as a gateway or proxy, received an invalid + response from the upstream server it accessed in attempting to + fulfill the request. + +10.5.4 503 Service Unavailable + + The server is currently unable to handle the request due to a + temporary overloading or maintenance of the server. The implication + is that this is a temporary condition which will be alleviated after + some delay. If known, the length of the delay MAY be indicated in a + Retry-After header. If no Retry-After is given, the client SHOULD + handle the response as it would for a 500 response. + + Note: The existence of the 503 status code does not imply that a + server must use it when becoming overloaded. Some servers may wish + to simply refuse the connection. + + + + +Fielding, et al. Standards Track [Page 70] + +RFC 2616 HTTP/1.1 June 1999 + + +10.5.5 504 Gateway Timeout + + The server, while acting as a gateway or proxy, did not receive a + timely response from the upstream server specified by the URI (e.g. + HTTP, FTP, LDAP) or some other auxiliary server (e.g. DNS) it needed + to access in attempting to complete the request. + + Note: Note to implementors: some deployed proxies are known to + return 400 or 500 when DNS lookups time out. + +10.5.6 505 HTTP Version Not Supported + + The server does not support, or refuses to support, the HTTP protocol + version that was used in the request message. The server is + indicating that it is unable or unwilling to complete the request + using the same major version as the client, as described in section + 3.1, other than with this error message. The response SHOULD contain + an entity describing why that version is not supported and what other + protocols are supported by that server. + +11 Access Authentication + + HTTP provides several OPTIONAL challenge-response authentication + mechanisms which can be used by a server to challenge a client + request and by a client to provide authentication information. The + general framework for access authentication, and the specification of + "basic" and "digest" authentication, are specified in "HTTP + Authentication: Basic and Digest Access Authentication" [43]. This + specification adopts the definitions of "challenge" and "credentials" + from that specification. + +12 Content Negotiation + + Most HTTP responses include an entity which contains information for + interpretation by a human user. Naturally, it is desirable to supply + the user with the "best available" entity corresponding to the + request. Unfortunately for servers and caches, not all users have the + same preferences for what is "best," and not all user agents are + equally capable of rendering all entity types. For that reason, HTTP + has provisions for several mechanisms for "content negotiation" -- + the process of selecting the best representation for a given response + when there are multiple representations available. + + Note: This is not called "format negotiation" because the + alternate representations may be of the same media type, but use + different capabilities of that type, be in different languages, + etc. + + + + +Fielding, et al. Standards Track [Page 71] + +RFC 2616 HTTP/1.1 June 1999 + + + Any response containing an entity-body MAY be subject to negotiation, + including error responses. + + There are two kinds of content negotiation which are possible in + HTTP: server-driven and agent-driven negotiation. These two kinds of + negotiation are orthogonal and thus may be used separately or in + combination. One method of combination, referred to as transparent + negotiation, occurs when a cache uses the agent-driven negotiation + information provided by the origin server in order to provide + server-driven negotiation for subsequent requests. + +12.1 Server-driven Negotiation + + If the selection of the best representation for a response is made by + an algorithm located at the server, it is called server-driven + negotiation. Selection is based on the available representations of + the response (the dimensions over which it can vary; e.g. language, + content-coding, etc.) and the contents of particular header fields in + the request message or on other information pertaining to the request + (such as the network address of the client). + + Server-driven negotiation is advantageous when the algorithm for + selecting from among the available representations is difficult to + describe to the user agent, or when the server desires to send its + "best guess" to the client along with the first response (hoping to + avoid the round-trip delay of a subsequent request if the "best + guess" is good enough for the user). In order to improve the server's + guess, the user agent MAY include request header fields (Accept, + Accept-Language, Accept-Encoding, etc.) which describe its + preferences for such a response. + + Server-driven negotiation has disadvantages: + + 1. It is impossible for the server to accurately determine what + might be "best" for any given user, since that would require + complete knowledge of both the capabilities of the user agent + and the intended use for the response (e.g., does the user want + to view it on screen or print it on paper?). + + 2. Having the user agent describe its capabilities in every + request can be both very inefficient (given that only a small + percentage of responses have multiple representations) and a + potential violation of the user's privacy. + + 3. It complicates the implementation of an origin server and the + algorithms for generating responses to a request. + + + + + +Fielding, et al. Standards Track [Page 72] + +RFC 2616 HTTP/1.1 June 1999 + + + 4. It may limit a public cache's ability to use the same response + for multiple user's requests. + + HTTP/1.1 includes the following request-header fields for enabling + server-driven negotiation through description of user agent + capabilities and user preferences: Accept (section 14.1), Accept- + Charset (section 14.2), Accept-Encoding (section 14.3), Accept- + Language (section 14.4), and User-Agent (section 14.43). However, an + origin server is not limited to these dimensions and MAY vary the + response based on any aspect of the request, including information + outside the request-header fields or within extension header fields + not defined by this specification. + + The Vary header field can be used to express the parameters the + server uses to select a representation that is subject to server- + driven negotiation. See section 13.6 for use of the Vary header field + by caches and section 14.44 for use of the Vary header field by + servers. + +12.2 Agent-driven Negotiation + + With agent-driven negotiation, selection of the best representation + for a response is performed by the user agent after receiving an + initial response from the origin server. Selection is based on a list + of the available representations of the response included within the + header fields or entity-body of the initial response, with each + representation identified by its own URI. Selection from among the + representations may be performed automatically (if the user agent is + capable of doing so) or manually by the user selecting from a + generated (possibly hypertext) menu. + + Agent-driven negotiation is advantageous when the response would vary + over commonly-used dimensions (such as type, language, or encoding), + when the origin server is unable to determine a user agent's + capabilities from examining the request, and generally when public + caches are used to distribute server load and reduce network usage. + + Agent-driven negotiation suffers from the disadvantage of needing a + second request to obtain the best alternate representation. This + second request is only efficient when caching is used. In addition, + this specification does not define any mechanism for supporting + automatic selection, though it also does not prevent any such + mechanism from being developed as an extension and used within + HTTP/1.1. + + + + + + + +Fielding, et al. Standards Track [Page 73] + +RFC 2616 HTTP/1.1 June 1999 + + + HTTP/1.1 defines the 300 (Multiple Choices) and 406 (Not Acceptable) + status codes for enabling agent-driven negotiation when the server is + unwilling or unable to provide a varying response using server-driven + negotiation. + +12.3 Transparent Negotiation + + Transparent negotiation is a combination of both server-driven and + agent-driven negotiation. When a cache is supplied with a form of the + list of available representations of the response (as in agent-driven + negotiation) and the dimensions of variance are completely understood + by the cache, then the cache becomes capable of performing server- + driven negotiation on behalf of the origin server for subsequent + requests on that resource. + + Transparent negotiation has the advantage of distributing the + negotiation work that would otherwise be required of the origin + server and also removing the second request delay of agent-driven + negotiation when the cache is able to correctly guess the right + response. + + This specification does not define any mechanism for transparent + negotiation, though it also does not prevent any such mechanism from + being developed as an extension that could be used within HTTP/1.1. + +13 Caching in HTTP + + HTTP is typically used for distributed information systems, where + performance can be improved by the use of response caches. The + HTTP/1.1 protocol includes a number of elements intended to make + caching work as well as possible. Because these elements are + inextricable from other aspects of the protocol, and because they + interact with each other, it is useful to describe the basic caching + design of HTTP separately from the detailed descriptions of methods, + headers, response codes, etc. + + Caching would be useless if it did not significantly improve + performance. The goal of caching in HTTP/1.1 is to eliminate the need + to send requests in many cases, and to eliminate the need to send + full responses in many other cases. The former reduces the number of + network round-trips required for many operations; we use an + "expiration" mechanism for this purpose (see section 13.2). The + latter reduces network bandwidth requirements; we use a "validation" + mechanism for this purpose (see section 13.3). + + Requirements for performance, availability, and disconnected + operation require us to be able to relax the goal of semantic + transparency. The HTTP/1.1 protocol allows origin servers, caches, + + + +Fielding, et al. Standards Track [Page 74] + +RFC 2616 HTTP/1.1 June 1999 + + + and clients to explicitly reduce transparency when necessary. + However, because non-transparent operation may confuse non-expert + users, and might be incompatible with certain server applications + (such as those for ordering merchandise), the protocol requires that + transparency be relaxed + + - only by an explicit protocol-level request when relaxed by + client or origin server + + - only with an explicit warning to the end user when relaxed by + cache or client + + Therefore, the HTTP/1.1 protocol provides these important elements: + + 1. Protocol features that provide full semantic transparency when + this is required by all parties. + + 2. Protocol features that allow an origin server or user agent to + explicitly request and control non-transparent operation. + + 3. Protocol features that allow a cache to attach warnings to + responses that do not preserve the requested approximation of + semantic transparency. + + A basic principle is that it must be possible for the clients to + detect any potential relaxation of semantic transparency. + + Note: The server, cache, or client implementor might be faced with + design decisions not explicitly discussed in this specification. + If a decision might affect semantic transparency, the implementor + ought to err on the side of maintaining transparency unless a + careful and complete analysis shows significant benefits in + breaking transparency. + +13.1.1 Cache Correctness + + A correct cache MUST respond to a request with the most up-to-date + response held by the cache that is appropriate to the request (see + sections 13.2.5, 13.2.6, and 13.12) which meets one of the following + conditions: + + 1. It has been checked for equivalence with what the origin server + would have returned by revalidating the response with the + origin server (section 13.3); + + + + + + + +Fielding, et al. Standards Track [Page 75] + +RFC 2616 HTTP/1.1 June 1999 + + + 2. It is "fresh enough" (see section 13.2). In the default case, + this means it meets the least restrictive freshness requirement + of the client, origin server, and cache (see section 14.9); if + the origin server so specifies, it is the freshness requirement + of the origin server alone. + + If a stored response is not "fresh enough" by the most + restrictive freshness requirement of both the client and the + origin server, in carefully considered circumstances the cache + MAY still return the response with the appropriate Warning + header (see section 13.1.5 and 14.46), unless such a response + is prohibited (e.g., by a "no-store" cache-directive, or by a + "no-cache" cache-request-directive; see section 14.9). + + 3. It is an appropriate 304 (Not Modified), 305 (Proxy Redirect), + or error (4xx or 5xx) response message. + + If the cache can not communicate with the origin server, then a + correct cache SHOULD respond as above if the response can be + correctly served from the cache; if not it MUST return an error or + warning indicating that there was a communication failure. + + If a cache receives a response (either an entire response, or a 304 + (Not Modified) response) that it would normally forward to the + requesting client, and the received response is no longer fresh, the + cache SHOULD forward it to the requesting client without adding a new + Warning (but without removing any existing Warning headers). A cache + SHOULD NOT attempt to revalidate a response simply because that + response became stale in transit; this might lead to an infinite + loop. A user agent that receives a stale response without a Warning + MAY display a warning indication to the user. + +13.1.2 Warnings + + Whenever a cache returns a response that is neither first-hand nor + "fresh enough" (in the sense of condition 2 in section 13.1.1), it + MUST attach a warning to that effect, using a Warning general-header. + The Warning header and the currently defined warnings are described + in section 14.46. The warning allows clients to take appropriate + action. + + Warnings MAY be used for other purposes, both cache-related and + otherwise. The use of a warning, rather than an error status code, + distinguish these responses from true failures. + + Warnings are assigned three digit warn-codes. The first digit + indicates whether the Warning MUST or MUST NOT be deleted from a + stored cache entry after a successful revalidation: + + + +Fielding, et al. Standards Track [Page 76] + +RFC 2616 HTTP/1.1 June 1999 + + + 1xx Warnings that describe the freshness or revalidation status of + the response, and so MUST be deleted after a successful + revalidation. 1XX warn-codes MAY be generated by a cache only when + validating a cached entry. It MUST NOT be generated by clients. + + 2xx Warnings that describe some aspect of the entity body or entity + headers that is not rectified by a revalidation (for example, a + lossy compression of the entity bodies) and which MUST NOT be + deleted after a successful revalidation. + + See section 14.46 for the definitions of the codes themselves. + + HTTP/1.0 caches will cache all Warnings in responses, without + deleting the ones in the first category. Warnings in responses that + are passed to HTTP/1.0 caches carry an extra warning-date field, + which prevents a future HTTP/1.1 recipient from believing an + erroneously cached Warning. + + Warnings also carry a warning text. The text MAY be in any + appropriate natural language (perhaps based on the client's Accept + headers), and include an OPTIONAL indication of what character set is + used. + + Multiple warnings MAY be attached to a response (either by the origin + server or by a cache), including multiple warnings with the same code + number. For example, a server might provide the same warning with + texts in both English and Basque. + + When multiple warnings are attached to a response, it might not be + practical or reasonable to display all of them to the user. This + version of HTTP does not specify strict priority rules for deciding + which warnings to display and in what order, but does suggest some + heuristics. + +13.1.3 Cache-control Mechanisms + + The basic cache mechanisms in HTTP/1.1 (server-specified expiration + times and validators) are implicit directives to caches. In some + cases, a server or client might need to provide explicit directives + to the HTTP caches. We use the Cache-Control header for this purpose. + + The Cache-Control header allows a client or server to transmit a + variety of directives in either requests or responses. These + directives typically override the default caching algorithms. As a + general rule, if there is any apparent conflict between header + values, the most restrictive interpretation is applied (that is, the + one that is most likely to preserve semantic transparency). However, + + + + +Fielding, et al. Standards Track [Page 77] + +RFC 2616 HTTP/1.1 June 1999 + + + in some cases, cache-control directives are explicitly specified as + weakening the approximation of semantic transparency (for example, + "max-stale" or "public"). + + The cache-control directives are described in detail in section 14.9. + +13.1.4 Explicit User Agent Warnings + + Many user agents make it possible for users to override the basic + caching mechanisms. For example, the user agent might allow the user + to specify that cached entities (even explicitly stale ones) are + never validated. Or the user agent might habitually add "Cache- + Control: max-stale=3600" to every request. The user agent SHOULD NOT + default to either non-transparent behavior, or behavior that results + in abnormally ineffective caching, but MAY be explicitly configured + to do so by an explicit action of the user. + + If the user has overridden the basic caching mechanisms, the user + agent SHOULD explicitly indicate to the user whenever this results in + the display of information that might not meet the server's + transparency requirements (in particular, if the displayed entity is + known to be stale). Since the protocol normally allows the user agent + to determine if responses are stale or not, this indication need only + be displayed when this actually happens. The indication need not be a + dialog box; it could be an icon (for example, a picture of a rotting + fish) or some other indicator. + + If the user has overridden the caching mechanisms in a way that would + abnormally reduce the effectiveness of caches, the user agent SHOULD + continually indicate this state to the user (for example, by a + display of a picture of currency in flames) so that the user does not + inadvertently consume excess resources or suffer from excessive + latency. + +13.1.5 Exceptions to the Rules and Warnings + + In some cases, the operator of a cache MAY choose to configure it to + return stale responses even when not requested by clients. This + decision ought not be made lightly, but may be necessary for reasons + of availability or performance, especially when the cache is poorly + connected to the origin server. Whenever a cache returns a stale + response, it MUST mark it as such (using a Warning header) enabling + the client software to alert the user that there might be a potential + problem. + + + + + + + +Fielding, et al. Standards Track [Page 78] + +RFC 2616 HTTP/1.1 June 1999 + + + It also allows the user agent to take steps to obtain a first-hand or + fresh response. For this reason, a cache SHOULD NOT return a stale + response if the client explicitly requests a first-hand or fresh one, + unless it is impossible to comply for technical or policy reasons. + +13.1.6 Client-controlled Behavior + + While the origin server (and to a lesser extent, intermediate caches, + by their contribution to the age of a response) are the primary + source of expiration information, in some cases the client might need + to control a cache's decision about whether to return a cached + response without validating it. Clients do this using several + directives of the Cache-Control header. + + A client's request MAY specify the maximum age it is willing to + accept of an unvalidated response; specifying a value of zero forces + the cache(s) to revalidate all responses. A client MAY also specify + the minimum time remaining before a response expires. Both of these + options increase constraints on the behavior of caches, and so cannot + further relax the cache's approximation of semantic transparency. + + A client MAY also specify that it will accept stale responses, up to + some maximum amount of staleness. This loosens the constraints on the + caches, and so might violate the origin server's specified + constraints on semantic transparency, but might be necessary to + support disconnected operation, or high availability in the face of + poor connectivity. + +13.2 Expiration Model + +13.2.1 Server-Specified Expiration + + HTTP caching works best when caches can entirely avoid making + requests to the origin server. The primary mechanism for avoiding + requests is for an origin server to provide an explicit expiration + time in the future, indicating that a response MAY be used to satisfy + subsequent requests. In other words, a cache can return a fresh + response without first contacting the server. + + Our expectation is that servers will assign future explicit + expiration times to responses in the belief that the entity is not + likely to change, in a semantically significant way, before the + expiration time is reached. This normally preserves semantic + transparency, as long as the server's expiration times are carefully + chosen. + + + + + + +Fielding, et al. Standards Track [Page 79] + +RFC 2616 HTTP/1.1 June 1999 + + + The expiration mechanism applies only to responses taken from a cache + and not to first-hand responses forwarded immediately to the + requesting client. + + If an origin server wishes to force a semantically transparent cache + to validate every request, it MAY assign an explicit expiration time + in the past. This means that the response is always stale, and so the + cache SHOULD validate it before using it for subsequent requests. See + section 14.9.4 for a more restrictive way to force revalidation. + + If an origin server wishes to force any HTTP/1.1 cache, no matter how + it is configured, to validate every request, it SHOULD use the "must- + revalidate" cache-control directive (see section 14.9). + + Servers specify explicit expiration times using either the Expires + header, or the max-age directive of the Cache-Control header. + + An expiration time cannot be used to force a user agent to refresh + its display or reload a resource; its semantics apply only to caching + mechanisms, and such mechanisms need only check a resource's + expiration status when a new request for that resource is initiated. + See section 13.13 for an explanation of the difference between caches + and history mechanisms. + +13.2.2 Heuristic Expiration + + Since origin servers do not always provide explicit expiration times, + HTTP caches typically assign heuristic expiration times, employing + algorithms that use other header values (such as the Last-Modified + time) to estimate a plausible expiration time. The HTTP/1.1 + specification does not provide specific algorithms, but does impose + worst-case constraints on their results. Since heuristic expiration + times might compromise semantic transparency, they ought to used + cautiously, and we encourage origin servers to provide explicit + expiration times as much as possible. + +13.2.3 Age Calculations + + In order to know if a cached entry is fresh, a cache needs to know if + its age exceeds its freshness lifetime. We discuss how to calculate + the latter in section 13.2.4; this section describes how to calculate + the age of a response or cache entry. + + In this discussion, we use the term "now" to mean "the current value + of the clock at the host performing the calculation." Hosts that use + HTTP, but especially hosts running origin servers and caches, SHOULD + use NTP [28] or some similar protocol to synchronize their clocks to + a globally accurate time standard. + + + +Fielding, et al. Standards Track [Page 80] + +RFC 2616 HTTP/1.1 June 1999 + + + HTTP/1.1 requires origin servers to send a Date header, if possible, + with every response, giving the time at which the response was + generated (see section 14.18). We use the term "date_value" to denote + the value of the Date header, in a form appropriate for arithmetic + operations. + + HTTP/1.1 uses the Age response-header to convey the estimated age of + the response message when obtained from a cache. The Age field value + is the cache's estimate of the amount of time since the response was + generated or revalidated by the origin server. + + In essence, the Age value is the sum of the time that the response + has been resident in each of the caches along the path from the + origin server, plus the amount of time it has been in transit along + network paths. + + We use the term "age_value" to denote the value of the Age header, in + a form appropriate for arithmetic operations. + + A response's age can be calculated in two entirely independent ways: + + 1. now minus date_value, if the local clock is reasonably well + synchronized to the origin server's clock. If the result is + negative, the result is replaced by zero. + + 2. age_value, if all of the caches along the response path + implement HTTP/1.1. + + Given that we have two independent ways to compute the age of a + response when it is received, we can combine these as + + corrected_received_age = max(now - date_value, age_value) + + and as long as we have either nearly synchronized clocks or all- + HTTP/1.1 paths, one gets a reliable (conservative) result. + + Because of network-imposed delays, some significant interval might + pass between the time that a server generates a response and the time + it is received at the next outbound cache or client. If uncorrected, + this delay could result in improperly low ages. + + Because the request that resulted in the returned Age value must have + been initiated prior to that Age value's generation, we can correct + for delays imposed by the network by recording the time at which the + request was initiated. Then, when an Age value is received, it MUST + be interpreted relative to the time the request was initiated, not + + + + + +Fielding, et al. Standards Track [Page 81] + +RFC 2616 HTTP/1.1 June 1999 + + + the time that the response was received. This algorithm results in + conservative behavior no matter how much delay is experienced. So, we + compute: + + corrected_initial_age = corrected_received_age + + (now - request_time) + + where "request_time" is the time (according to the local clock) when + the request that elicited this response was sent. + + Summary of age calculation algorithm, when a cache receives a + response: + + /* + * age_value + * is the value of Age: header received by the cache with + * this response. + * date_value + * is the value of the origin server's Date: header + * request_time + * is the (local) time when the cache made the request + * that resulted in this cached response + * response_time + * is the (local) time when the cache received the + * response + * now + * is the current (local) time + */ + + apparent_age = max(0, response_time - date_value); + corrected_received_age = max(apparent_age, age_value); + response_delay = response_time - request_time; + corrected_initial_age = corrected_received_age + response_delay; + resident_time = now - response_time; + current_age = corrected_initial_age + resident_time; + + The current_age of a cache entry is calculated by adding the amount + of time (in seconds) since the cache entry was last validated by the + origin server to the corrected_initial_age. When a response is + generated from a cache entry, the cache MUST include a single Age + header field in the response with a value equal to the cache entry's + current_age. + + The presence of an Age header field in a response implies that a + response is not first-hand. However, the converse is not true, since + the lack of an Age header field in a response does not imply that the + + + + + +Fielding, et al. Standards Track [Page 82] + +RFC 2616 HTTP/1.1 June 1999 + + + response is first-hand unless all caches along the request path are + compliant with HTTP/1.1 (i.e., older HTTP caches did not implement + the Age header field). + +13.2.4 Expiration Calculations + + In order to decide whether a response is fresh or stale, we need to + compare its freshness lifetime to its age. The age is calculated as + described in section 13.2.3; this section describes how to calculate + the freshness lifetime, and to determine if a response has expired. + In the discussion below, the values can be represented in any form + appropriate for arithmetic operations. + + We use the term "expires_value" to denote the value of the Expires + header. We use the term "max_age_value" to denote an appropriate + value of the number of seconds carried by the "max-age" directive of + the Cache-Control header in a response (see section 14.9.3). + + The max-age directive takes priority over Expires, so if max-age is + present in a response, the calculation is simply: + + freshness_lifetime = max_age_value + + Otherwise, if Expires is present in the response, the calculation is: + + freshness_lifetime = expires_value - date_value + + Note that neither of these calculations is vulnerable to clock skew, + since all of the information comes from the origin server. + + If none of Expires, Cache-Control: max-age, or Cache-Control: s- + maxage (see section 14.9.3) appears in the response, and the response + does not include other restrictions on caching, the cache MAY compute + a freshness lifetime using a heuristic. The cache MUST attach Warning + 113 to any response whose age is more than 24 hours if such warning + has not already been added. + + Also, if the response does have a Last-Modified time, the heuristic + expiration value SHOULD be no more than some fraction of the interval + since that time. A typical setting of this fraction might be 10%. + + The calculation to determine if a response has expired is quite + simple: + + response_is_fresh = (freshness_lifetime > current_age) + + + + + + +Fielding, et al. Standards Track [Page 83] + +RFC 2616 HTTP/1.1 June 1999 + + +13.2.5 Disambiguating Expiration Values + + Because expiration values are assigned optimistically, it is possible + for two caches to contain fresh values for the same resource that are + different. + + If a client performing a retrieval receives a non-first-hand response + for a request that was already fresh in its own cache, and the Date + header in its existing cache entry is newer than the Date on the new + response, then the client MAY ignore the response. If so, it MAY + retry the request with a "Cache-Control: max-age=0" directive (see + section 14.9), to force a check with the origin server. + + If a cache has two fresh responses for the same representation with + different validators, it MUST use the one with the more recent Date + header. This situation might arise because the cache is pooling + responses from other caches, or because a client has asked for a + reload or a revalidation of an apparently fresh cache entry. + +13.2.6 Disambiguating Multiple Responses + + Because a client might be receiving responses via multiple paths, so + that some responses flow through one set of caches and other + responses flow through a different set of caches, a client might + receive responses in an order different from that in which the origin + server sent them. We would like the client to use the most recently + generated response, even if older responses are still apparently + fresh. + + Neither the entity tag nor the expiration value can impose an + ordering on responses, since it is possible that a later response + intentionally carries an earlier expiration time. The Date values are + ordered to a granularity of one second. + + When a client tries to revalidate a cache entry, and the response it + receives contains a Date header that appears to be older than the one + for the existing entry, then the client SHOULD repeat the request + unconditionally, and include + + Cache-Control: max-age=0 + + to force any intermediate caches to validate their copies directly + with the origin server, or + + Cache-Control: no-cache + + to force any intermediate caches to obtain a new copy from the origin + server. + + + +Fielding, et al. Standards Track [Page 84] + +RFC 2616 HTTP/1.1 June 1999 + + + If the Date values are equal, then the client MAY use either response + (or MAY, if it is being extremely prudent, request a new response). + Servers MUST NOT depend on clients being able to choose + deterministically between responses generated during the same second, + if their expiration times overlap. + +13.3 Validation Model + + When a cache has a stale entry that it would like to use as a + response to a client's request, it first has to check with the origin + server (or possibly an intermediate cache with a fresh response) to + see if its cached entry is still usable. We call this "validating" + the cache entry. Since we do not want to have to pay the overhead of + retransmitting the full response if the cached entry is good, and we + do not want to pay the overhead of an extra round trip if the cached + entry is invalid, the HTTP/1.1 protocol supports the use of + conditional methods. + + The key protocol features for supporting conditional methods are + those concerned with "cache validators." When an origin server + generates a full response, it attaches some sort of validator to it, + which is kept with the cache entry. When a client (user agent or + proxy cache) makes a conditional request for a resource for which it + has a cache entry, it includes the associated validator in the + request. + + The server then checks that validator against the current validator + for the entity, and, if they match (see section 13.3.3), it responds + with a special status code (usually, 304 (Not Modified)) and no + entity-body. Otherwise, it returns a full response (including + entity-body). Thus, we avoid transmitting the full response if the + validator matches, and we avoid an extra round trip if it does not + match. + + In HTTP/1.1, a conditional request looks exactly the same as a normal + request for the same resource, except that it carries a special + header (which includes the validator) that implicitly turns the + method (usually, GET) into a conditional. + + The protocol includes both positive and negative senses of cache- + validating conditions. That is, it is possible to request either that + a method be performed if and only if a validator matches or if and + only if no validators match. + + + + + + + + +Fielding, et al. Standards Track [Page 85] + +RFC 2616 HTTP/1.1 June 1999 + + + Note: a response that lacks a validator may still be cached, and + served from cache until it expires, unless this is explicitly + prohibited by a cache-control directive. However, a cache cannot + do a conditional retrieval if it does not have a validator for the + entity, which means it will not be refreshable after it expires. + +13.3.1 Last-Modified Dates + + The Last-Modified entity-header field value is often used as a cache + validator. In simple terms, a cache entry is considered to be valid + if the entity has not been modified since the Last-Modified value. + +13.3.2 Entity Tag Cache Validators + + The ETag response-header field value, an entity tag, provides for an + "opaque" cache validator. This might allow more reliable validation + in situations where it is inconvenient to store modification dates, + where the one-second resolution of HTTP date values is not + sufficient, or where the origin server wishes to avoid certain + paradoxes that might arise from the use of modification dates. + + Entity Tags are described in section 3.11. The headers used with + entity tags are described in sections 14.19, 14.24, 14.26 and 14.44. + +13.3.3 Weak and Strong Validators + + Since both origin servers and caches will compare two validators to + decide if they represent the same or different entities, one normally + would expect that if the entity (the entity-body or any entity- + headers) changes in any way, then the associated validator would + change as well. If this is true, then we call this validator a + "strong validator." + + However, there might be cases when a server prefers to change the + validator only on semantically significant changes, and not when + insignificant aspects of the entity change. A validator that does not + always change when the resource changes is a "weak validator." + + Entity tags are normally "strong validators," but the protocol + provides a mechanism to tag an entity tag as "weak." One can think of + a strong validator as one that changes whenever the bits of an entity + changes, while a weak value changes whenever the meaning of an entity + changes. Alternatively, one can think of a strong validator as part + of an identifier for a specific entity, while a weak validator is + part of an identifier for a set of semantically equivalent entities. + + Note: One example of a strong validator is an integer that is + incremented in stable storage every time an entity is changed. + + + +Fielding, et al. Standards Track [Page 86] + +RFC 2616 HTTP/1.1 June 1999 + + + An entity's modification time, if represented with one-second + resolution, could be a weak validator, since it is possible that + the resource might be modified twice during a single second. + + Support for weak validators is optional. However, weak validators + allow for more efficient caching of equivalent objects; for + example, a hit counter on a site is probably good enough if it is + updated every few days or weeks, and any value during that period + is likely "good enough" to be equivalent. + + A "use" of a validator is either when a client generates a request + and includes the validator in a validating header field, or when a + server compares two validators. + + Strong validators are usable in any context. Weak validators are only + usable in contexts that do not depend on exact equality of an entity. + For example, either kind is usable for a conditional GET of a full + entity. However, only a strong validator is usable for a sub-range + retrieval, since otherwise the client might end up with an internally + inconsistent entity. + + Clients MAY issue simple (non-subrange) GET requests with either weak + validators or strong validators. Clients MUST NOT use weak validators + in other forms of request. + + The only function that the HTTP/1.1 protocol defines on validators is + comparison. There are two validator comparison functions, depending + on whether the comparison context allows the use of weak validators + or not: + + - The strong comparison function: in order to be considered equal, + both validators MUST be identical in every way, and both MUST + NOT be weak. + + - The weak comparison function: in order to be considered equal, + both validators MUST be identical in every way, but either or + both of them MAY be tagged as "weak" without affecting the + result. + + An entity tag is strong unless it is explicitly tagged as weak. + Section 3.11 gives the syntax for entity tags. + + A Last-Modified time, when used as a validator in a request, is + implicitly weak unless it is possible to deduce that it is strong, + using the following rules: + + - The validator is being compared by an origin server to the + actual current validator for the entity and, + + + +Fielding, et al. Standards Track [Page 87] + +RFC 2616 HTTP/1.1 June 1999 + + + - That origin server reliably knows that the associated entity did + not change twice during the second covered by the presented + validator. + + or + + - The validator is about to be used by a client in an If- + Modified-Since or If-Unmodified-Since header, because the client + has a cache entry for the associated entity, and + + - That cache entry includes a Date value, which gives the time + when the origin server sent the original response, and + + - The presented Last-Modified time is at least 60 seconds before + the Date value. + + or + + - The validator is being compared by an intermediate cache to the + validator stored in its cache entry for the entity, and + + - That cache entry includes a Date value, which gives the time + when the origin server sent the original response, and + + - The presented Last-Modified time is at least 60 seconds before + the Date value. + + This method relies on the fact that if two different responses were + sent by the origin server during the same second, but both had the + same Last-Modified time, then at least one of those responses would + have a Date value equal to its Last-Modified time. The arbitrary 60- + second limit guards against the possibility that the Date and Last- + Modified values are generated from different clocks, or at somewhat + different times during the preparation of the response. An + implementation MAY use a value larger than 60 seconds, if it is + believed that 60 seconds is too short. + + If a client wishes to perform a sub-range retrieval on a value for + which it has only a Last-Modified time and no opaque validator, it + MAY do this only if the Last-Modified time is strong in the sense + described here. + + A cache or origin server receiving a conditional request, other than + a full-body GET request, MUST use the strong comparison function to + evaluate the condition. + + These rules allow HTTP/1.1 caches and clients to safely perform sub- + range retrievals on values that have been obtained from HTTP/1.0 + + + +Fielding, et al. Standards Track [Page 88] + +RFC 2616 HTTP/1.1 June 1999 + + + servers. + +13.3.4 Rules for When to Use Entity Tags and Last-Modified Dates + + We adopt a set of rules and recommendations for origin servers, + clients, and caches regarding when various validator types ought to + be used, and for what purposes. + + HTTP/1.1 origin servers: + + - SHOULD send an entity tag validator unless it is not feasible to + generate one. + + - MAY send a weak entity tag instead of a strong entity tag, if + performance considerations support the use of weak entity tags, + or if it is unfeasible to send a strong entity tag. + + - SHOULD send a Last-Modified value if it is feasible to send one, + unless the risk of a breakdown in semantic transparency that + could result from using this date in an If-Modified-Since header + would lead to serious problems. + + In other words, the preferred behavior for an HTTP/1.1 origin server + is to send both a strong entity tag and a Last-Modified value. + + In order to be legal, a strong entity tag MUST change whenever the + associated entity value changes in any way. A weak entity tag SHOULD + change whenever the associated entity changes in a semantically + significant way. + + Note: in order to provide semantically transparent caching, an + origin server must avoid reusing a specific strong entity tag + value for two different entities, or reusing a specific weak + entity tag value for two semantically different entities. Cache + entries might persist for arbitrarily long periods, regardless of + expiration times, so it might be inappropriate to expect that a + cache will never again attempt to validate an entry using a + validator that it obtained at some point in the past. + + HTTP/1.1 clients: + + - If an entity tag has been provided by the origin server, MUST + use that entity tag in any cache-conditional request (using If- + Match or If-None-Match). + + - If only a Last-Modified value has been provided by the origin + server, SHOULD use that value in non-subrange cache-conditional + requests (using If-Modified-Since). + + + +Fielding, et al. Standards Track [Page 89] + +RFC 2616 HTTP/1.1 June 1999 + + + - If only a Last-Modified value has been provided by an HTTP/1.0 + origin server, MAY use that value in subrange cache-conditional + requests (using If-Unmodified-Since:). The user agent SHOULD + provide a way to disable this, in case of difficulty. + + - If both an entity tag and a Last-Modified value have been + provided by the origin server, SHOULD use both validators in + cache-conditional requests. This allows both HTTP/1.0 and + HTTP/1.1 caches to respond appropriately. + + An HTTP/1.1 origin server, upon receiving a conditional request that + includes both a Last-Modified date (e.g., in an If-Modified-Since or + If-Unmodified-Since header field) and one or more entity tags (e.g., + in an If-Match, If-None-Match, or If-Range header field) as cache + validators, MUST NOT return a response status of 304 (Not Modified) + unless doing so is consistent with all of the conditional header + fields in the request. + + An HTTP/1.1 caching proxy, upon receiving a conditional request that + includes both a Last-Modified date and one or more entity tags as + cache validators, MUST NOT return a locally cached response to the + client unless that cached response is consistent with all of the + conditional header fields in the request. + + Note: The general principle behind these rules is that HTTP/1.1 + servers and clients should transmit as much non-redundant + information as is available in their responses and requests. + HTTP/1.1 systems receiving this information will make the most + conservative assumptions about the validators they receive. + + HTTP/1.0 clients and caches will ignore entity tags. Generally, + last-modified values received or used by these systems will + support transparent and efficient caching, and so HTTP/1.1 origin + servers should provide Last-Modified values. In those rare cases + where the use of a Last-Modified value as a validator by an + HTTP/1.0 system could result in a serious problem, then HTTP/1.1 + origin servers should not provide one. + +13.3.5 Non-validating Conditionals + + The principle behind entity tags is that only the service author + knows the semantics of a resource well enough to select an + appropriate cache validation mechanism, and the specification of any + validator comparison function more complex than byte-equality would + open up a can of worms. Thus, comparisons of any other headers + (except Last-Modified, for compatibility with HTTP/1.0) are never + used for purposes of validating a cache entry. + + + + +Fielding, et al. Standards Track [Page 90] + +RFC 2616 HTTP/1.1 June 1999 + + +13.4 Response Cacheability + + Unless specifically constrained by a cache-control (section 14.9) + directive, a caching system MAY always store a successful response + (see section 13.8) as a cache entry, MAY return it without validation + if it is fresh, and MAY return it after successful validation. If + there is neither a cache validator nor an explicit expiration time + associated with a response, we do not expect it to be cached, but + certain caches MAY violate this expectation (for example, when little + or no network connectivity is available). A client can usually detect + that such a response was taken from a cache by comparing the Date + header to the current time. + + Note: some HTTP/1.0 caches are known to violate this expectation + without providing any Warning. + + However, in some cases it might be inappropriate for a cache to + retain an entity, or to return it in response to a subsequent + request. This might be because absolute semantic transparency is + deemed necessary by the service author, or because of security or + privacy considerations. Certain cache-control directives are + therefore provided so that the server can indicate that certain + resource entities, or portions thereof, are not to be cached + regardless of other considerations. + + Note that section 14.8 normally prevents a shared cache from saving + and returning a response to a previous request if that request + included an Authorization header. + + A response received with a status code of 200, 203, 206, 300, 301 or + 410 MAY be stored by a cache and used in reply to a subsequent + request, subject to the expiration mechanism, unless a cache-control + directive prohibits caching. However, a cache that does not support + the Range and Content-Range headers MUST NOT cache 206 (Partial + Content) responses. + + A response received with any other status code (e.g. status codes 302 + and 307) MUST NOT be returned in a reply to a subsequent request + unless there are cache-control directives or another header(s) that + explicitly allow it. For example, these include the following: an + Expires header (section 14.21); a "max-age", "s-maxage", "must- + revalidate", "proxy-revalidate", "public" or "private" cache-control + directive (section 14.9). + + + + + + + + +Fielding, et al. Standards Track [Page 91] + +RFC 2616 HTTP/1.1 June 1999 + + +13.5 Constructing Responses From Caches + + The purpose of an HTTP cache is to store information received in + response to requests for use in responding to future requests. In + many cases, a cache simply returns the appropriate parts of a + response to the requester. However, if the cache holds a cache entry + based on a previous response, it might have to combine parts of a new + response with what is held in the cache entry. + +13.5.1 End-to-end and Hop-by-hop Headers + + For the purpose of defining the behavior of caches and non-caching + proxies, we divide HTTP headers into two categories: + + - End-to-end headers, which are transmitted to the ultimate + recipient of a request or response. End-to-end headers in + responses MUST be stored as part of a cache entry and MUST be + transmitted in any response formed from a cache entry. + + - Hop-by-hop headers, which are meaningful only for a single + transport-level connection, and are not stored by caches or + forwarded by proxies. + + The following HTTP/1.1 headers are hop-by-hop headers: + + - Connection + - Keep-Alive + - Proxy-Authenticate + - Proxy-Authorization + - TE + - Trailers + - Transfer-Encoding + - Upgrade + + All other headers defined by HTTP/1.1 are end-to-end headers. + + Other hop-by-hop headers MUST be listed in a Connection header, + (section 14.10) to be introduced into HTTP/1.1 (or later). + +13.5.2 Non-modifiable Headers + + Some features of the HTTP/1.1 protocol, such as Digest + Authentication, depend on the value of certain end-to-end headers. A + transparent proxy SHOULD NOT modify an end-to-end header unless the + definition of that header requires or specifically allows that. + + + + + + +Fielding, et al. Standards Track [Page 92] + +RFC 2616 HTTP/1.1 June 1999 + + + A transparent proxy MUST NOT modify any of the following fields in a + request or response, and it MUST NOT add any of these fields if not + already present: + + - Content-Location + + - Content-MD5 + + - ETag + + - Last-Modified + + A transparent proxy MUST NOT modify any of the following fields in a + response: + + - Expires + + but it MAY add any of these fields if not already present. If an + Expires header is added, it MUST be given a field-value identical to + that of the Date header in that response. + + A proxy MUST NOT modify or add any of the following fields in a + message that contains the no-transform cache-control directive, or in + any request: + + - Content-Encoding + + - Content-Range + + - Content-Type + + A non-transparent proxy MAY modify or add these fields to a message + that does not include no-transform, but if it does so, it MUST add a + Warning 214 (Transformation applied) if one does not already appear + in the message (see section 14.46). + + Warning: unnecessary modification of end-to-end headers might + cause authentication failures if stronger authentication + mechanisms are introduced in later versions of HTTP. Such + authentication mechanisms MAY rely on the values of header fields + not listed here. + + The Content-Length field of a request or response is added or deleted + according to the rules in section 4.4. A transparent proxy MUST + preserve the entity-length (section 7.2.2) of the entity-body, + although it MAY change the transfer-length (section 4.4). + + + + + +Fielding, et al. Standards Track [Page 93] + +RFC 2616 HTTP/1.1 June 1999 + + +13.5.3 Combining Headers + + When a cache makes a validating request to a server, and the server + provides a 304 (Not Modified) response or a 206 (Partial Content) + response, the cache then constructs a response to send to the + requesting client. + + If the status code is 304 (Not Modified), the cache uses the entity- + body stored in the cache entry as the entity-body of this outgoing + response. If the status code is 206 (Partial Content) and the ETag or + Last-Modified headers match exactly, the cache MAY combine the + contents stored in the cache entry with the new contents received in + the response and use the result as the entity-body of this outgoing + response, (see 13.5.4). + + The end-to-end headers stored in the cache entry are used for the + constructed response, except that + + - any stored Warning headers with warn-code 1xx (see section + 14.46) MUST be deleted from the cache entry and the forwarded + response. + + - any stored Warning headers with warn-code 2xx MUST be retained + in the cache entry and the forwarded response. + + - any end-to-end headers provided in the 304 or 206 response MUST + replace the corresponding headers from the cache entry. + + Unless the cache decides to remove the cache entry, it MUST also + replace the end-to-end headers stored with the cache entry with + corresponding headers received in the incoming response, except for + Warning headers as described immediately above. If a header field- + name in the incoming response matches more than one header in the + cache entry, all such old headers MUST be replaced. + + In other words, the set of end-to-end headers received in the + incoming response overrides all corresponding end-to-end headers + stored with the cache entry (except for stored Warning headers with + warn-code 1xx, which are deleted even if not overridden). + + Note: this rule allows an origin server to use a 304 (Not + Modified) or a 206 (Partial Content) response to update any header + associated with a previous response for the same entity or sub- + ranges thereof, although it might not always be meaningful or + correct to do so. This rule does not allow an origin server to use + a 304 (Not Modified) or a 206 (Partial Content) response to + entirely delete a header that it had provided with a previous + response. + + + +Fielding, et al. Standards Track [Page 94] + +RFC 2616 HTTP/1.1 June 1999 + + +13.5.4 Combining Byte Ranges + + A response might transfer only a subrange of the bytes of an entity- + body, either because the request included one or more Range + specifications, or because a connection was broken prematurely. After + several such transfers, a cache might have received several ranges of + the same entity-body. + + If a cache has a stored non-empty set of subranges for an entity, and + an incoming response transfers another subrange, the cache MAY + combine the new subrange with the existing set if both the following + conditions are met: + + - Both the incoming response and the cache entry have a cache + validator. + + - The two cache validators match using the strong comparison + function (see section 13.3.3). + + If either requirement is not met, the cache MUST use only the most + recent partial response (based on the Date values transmitted with + every response, and using the incoming response if these values are + equal or missing), and MUST discard the other partial information. + +13.6 Caching Negotiated Responses + + Use of server-driven content negotiation (section 12.1), as indicated + by the presence of a Vary header field in a response, alters the + conditions and procedure by which a cache can use the response for + subsequent requests. See section 14.44 for use of the Vary header + field by servers. + + A server SHOULD use the Vary header field to inform a cache of what + request-header fields were used to select among multiple + representations of a cacheable response subject to server-driven + negotiation. The set of header fields named by the Vary field value + is known as the "selecting" request-headers. + + When the cache receives a subsequent request whose Request-URI + specifies one or more cache entries including a Vary header field, + the cache MUST NOT use such a cache entry to construct a response to + the new request unless all of the selecting request-headers present + in the new request match the corresponding stored request-headers in + the original request. + + The selecting request-headers from two requests are defined to match + if and only if the selecting request-headers in the first request can + be transformed to the selecting request-headers in the second request + + + +Fielding, et al. Standards Track [Page 95] + +RFC 2616 HTTP/1.1 June 1999 + + + by adding or removing linear white space (LWS) at places where this + is allowed by the corresponding BNF, and/or combining multiple + message-header fields with the same field name following the rules + about message headers in section 4.2. + + A Vary header field-value of "*" always fails to match and subsequent + requests on that resource can only be properly interpreted by the + origin server. + + If the selecting request header fields for the cached entry do not + match the selecting request header fields of the new request, then + the cache MUST NOT use a cached entry to satisfy the request unless + it first relays the new request to the origin server in a conditional + request and the server responds with 304 (Not Modified), including an + entity tag or Content-Location that indicates the entity to be used. + + If an entity tag was assigned to a cached representation, the + forwarded request SHOULD be conditional and include the entity tags + in an If-None-Match header field from all its cache entries for the + resource. This conveys to the server the set of entities currently + held by the cache, so that if any one of these entities matches the + requested entity, the server can use the ETag header field in its 304 + (Not Modified) response to tell the cache which entry is appropriate. + If the entity-tag of the new response matches that of an existing + entry, the new response SHOULD be used to update the header fields of + the existing entry, and the result MUST be returned to the client. + + If any of the existing cache entries contains only partial content + for the associated entity, its entity-tag SHOULD NOT be included in + the If-None-Match header field unless the request is for a range that + would be fully satisfied by that entry. + + If a cache receives a successful response whose Content-Location + field matches that of an existing cache entry for the same Request- + ]URI, whose entity-tag differs from that of the existing entry, and + whose Date is more recent than that of the existing entry, the + existing entry SHOULD NOT be returned in response to future requests + and SHOULD be deleted from the cache. + +13.7 Shared and Non-Shared Caches + + For reasons of security and privacy, it is necessary to make a + distinction between "shared" and "non-shared" caches. A non-shared + cache is one that is accessible only to a single user. Accessibility + in this case SHOULD be enforced by appropriate security mechanisms. + All other caches are considered to be "shared." Other sections of + + + + + +Fielding, et al. Standards Track [Page 96] + +RFC 2616 HTTP/1.1 June 1999 + + + this specification place certain constraints on the operation of + shared caches in order to prevent loss of privacy or failure of + access controls. + +13.8 Errors or Incomplete Response Cache Behavior + + A cache that receives an incomplete response (for example, with fewer + bytes of data than specified in a Content-Length header) MAY store + the response. However, the cache MUST treat this as a partial + response. Partial responses MAY be combined as described in section + 13.5.4; the result might be a full response or might still be + partial. A cache MUST NOT return a partial response to a client + without explicitly marking it as such, using the 206 (Partial + Content) status code. A cache MUST NOT return a partial response + using a status code of 200 (OK). + + If a cache receives a 5xx response while attempting to revalidate an + entry, it MAY either forward this response to the requesting client, + or act as if the server failed to respond. In the latter case, it MAY + return a previously received response unless the cached entry + includes the "must-revalidate" cache-control directive (see section + 14.9). + +13.9 Side Effects of GET and HEAD + + Unless the origin server explicitly prohibits the caching of their + responses, the application of GET and HEAD methods to any resources + SHOULD NOT have side effects that would lead to erroneous behavior if + these responses are taken from a cache. They MAY still have side + effects, but a cache is not required to consider such side effects in + its caching decisions. Caches are always expected to observe an + origin server's explicit restrictions on caching. + + We note one exception to this rule: since some applications have + traditionally used GETs and HEADs with query URLs (those containing a + "?" in the rel_path part) to perform operations with significant side + effects, caches MUST NOT treat responses to such URIs as fresh unless + the server provides an explicit expiration time. This specifically + means that responses from HTTP/1.0 servers for such URIs SHOULD NOT + be taken from a cache. See section 9.1.1 for related information. + +13.10 Invalidation After Updates or Deletions + + The effect of certain methods performed on a resource at the origin + server might cause one or more existing cache entries to become non- + transparently invalid. That is, although they might continue to be + "fresh," they do not accurately reflect what the origin server would + return for a new request on that resource. + + + +Fielding, et al. Standards Track [Page 97] + +RFC 2616 HTTP/1.1 June 1999 + + + There is no way for the HTTP protocol to guarantee that all such + cache entries are marked invalid. For example, the request that + caused the change at the origin server might not have gone through + the proxy where a cache entry is stored. However, several rules help + reduce the likelihood of erroneous behavior. + + In this section, the phrase "invalidate an entity" means that the + cache will either remove all instances of that entity from its + storage, or will mark these as "invalid" and in need of a mandatory + revalidation before they can be returned in response to a subsequent + request. + + Some HTTP methods MUST cause a cache to invalidate an entity. This is + either the entity referred to by the Request-URI, or by the Location + or Content-Location headers (if present). These methods are: + + - PUT + + - DELETE + + - POST + + In order to prevent denial of service attacks, an invalidation based + on the URI in a Location or Content-Location header MUST only be + performed if the host part is the same as in the Request-URI. + + A cache that passes through requests for methods it does not + understand SHOULD invalidate any entities referred to by the + Request-URI. + +13.11 Write-Through Mandatory + + All methods that might be expected to cause modifications to the + origin server's resources MUST be written through to the origin + server. This currently includes all methods except for GET and HEAD. + A cache MUST NOT reply to such a request from a client before having + transmitted the request to the inbound server, and having received a + corresponding response from the inbound server. This does not prevent + a proxy cache from sending a 100 (Continue) response before the + inbound server has sent its final reply. + + The alternative (known as "write-back" or "copy-back" caching) is not + allowed in HTTP/1.1, due to the difficulty of providing consistent + updates and the problems arising from server, cache, or network + failure prior to write-back. + + + + + + +Fielding, et al. Standards Track [Page 98] + +RFC 2616 HTTP/1.1 June 1999 + + +13.12 Cache Replacement + + If a new cacheable (see sections 14.9.2, 13.2.5, 13.2.6 and 13.8) + response is received from a resource while any existing responses for + the same resource are cached, the cache SHOULD use the new response + to reply to the current request. It MAY insert it into cache storage + and MAY, if it meets all other requirements, use it to respond to any + future requests that would previously have caused the old response to + be returned. If it inserts the new response into cache storage the + rules in section 13.5.3 apply. + + Note: a new response that has an older Date header value than + existing cached responses is not cacheable. + +13.13 History Lists + + User agents often have history mechanisms, such as "Back" buttons and + history lists, which can be used to redisplay an entity retrieved + earlier in a session. + + History mechanisms and caches are different. In particular history + mechanisms SHOULD NOT try to show a semantically transparent view of + the current state of a resource. Rather, a history mechanism is meant + to show exactly what the user saw at the time when the resource was + retrieved. + + By default, an expiration time does not apply to history mechanisms. + If the entity is still in storage, a history mechanism SHOULD display + it even if the entity has expired, unless the user has specifically + configured the agent to refresh expired history documents. + + This is not to be construed to prohibit the history mechanism from + telling the user that a view might be stale. + + Note: if history list mechanisms unnecessarily prevent users from + viewing stale resources, this will tend to force service authors + to avoid using HTTP expiration controls and cache controls when + they would otherwise like to. Service authors may consider it + important that users not be presented with error messages or + warning messages when they use navigation controls (such as BACK) + to view previously fetched resources. Even though sometimes such + resources ought not to cached, or ought to expire quickly, user + interface considerations may force service authors to resort to + other means of preventing caching (e.g. "once-only" URLs) in order + not to suffer the effects of improperly functioning history + mechanisms. + + + + + +Fielding, et al. Standards Track [Page 99] + +RFC 2616 HTTP/1.1 June 1999 + + +14 Header Field Definitions + + This section defines the syntax and semantics of all standard + HTTP/1.1 header fields. For entity-header fields, both sender and + recipient refer to either the client or the server, depending on who + sends and who receives the entity. + +14.1 Accept + + The Accept request-header field can be used to specify certain media + types which are acceptable for the response. Accept headers can be + used to indicate that the request is specifically limited to a small + set of desired types, as in the case of a request for an in-line + image. + + Accept = "Accept" ":" + #( media-range [ accept-params ] ) + + media-range = ( "*/*" + | ( type "/" "*" ) + | ( type "/" subtype ) + ) *( ";" parameter ) + accept-params = ";" "q" "=" qvalue *( accept-extension ) + accept-extension = ";" token [ "=" ( token | quoted-string ) ] + + The asterisk "*" character is used to group media types into ranges, + with "*/*" indicating all media types and "type/*" indicating all + subtypes of that type. The media-range MAY include media type + parameters that are applicable to that range. + + Each media-range MAY be followed by one or more accept-params, + beginning with the "q" parameter for indicating a relative quality + factor. The first "q" parameter (if any) separates the media-range + parameter(s) from the accept-params. Quality factors allow the user + or user agent to indicate the relative degree of preference for that + media-range, using the qvalue scale from 0 to 1 (section 3.9). The + default value is q=1. + + Note: Use of the "q" parameter name to separate media type + parameters from Accept extension parameters is due to historical + practice. Although this prevents any media type parameter named + "q" from being used with a media range, such an event is believed + to be unlikely given the lack of any "q" parameters in the IANA + media type registry and the rare usage of any media type + parameters in Accept. Future media types are discouraged from + registering any parameter named "q". + + + + + +Fielding, et al. Standards Track [Page 100] + +RFC 2616 HTTP/1.1 June 1999 + + + The example + + Accept: audio/*; q=0.2, audio/basic + + SHOULD be interpreted as "I prefer audio/basic, but send me any audio + type if it is the best available after an 80% mark-down in quality." + + If no Accept header field is present, then it is assumed that the + client accepts all media types. If an Accept header field is present, + and if the server cannot send a response which is acceptable + according to the combined Accept field value, then the server SHOULD + send a 406 (not acceptable) response. + + A more elaborate example is + + Accept: text/plain; q=0.5, text/html, + text/x-dvi; q=0.8, text/x-c + + Verbally, this would be interpreted as "text/html and text/x-c are + the preferred media types, but if they do not exist, then send the + text/x-dvi entity, and if that does not exist, send the text/plain + entity." + + Media ranges can be overridden by more specific media ranges or + specific media types. If more than one media range applies to a given + type, the most specific reference has precedence. For example, + + Accept: text/*, text/html, text/html;level=1, */* + + have the following precedence: + + 1) text/html;level=1 + 2) text/html + 3) text/* + 4) */* + + The media type quality factor associated with a given type is + determined by finding the media range with the highest precedence + which matches that type. For example, + + Accept: text/*;q=0.3, text/html;q=0.7, text/html;level=1, + text/html;level=2;q=0.4, */*;q=0.5 + + would cause the following values to be associated: + + text/html;level=1 = 1 + text/html = 0.7 + text/plain = 0.3 + + + +Fielding, et al. Standards Track [Page 101] + +RFC 2616 HTTP/1.1 June 1999 + + + image/jpeg = 0.5 + text/html;level=2 = 0.4 + text/html;level=3 = 0.7 + + Note: A user agent might be provided with a default set of quality + values for certain media ranges. However, unless the user agent is + a closed system which cannot interact with other rendering agents, + this default set ought to be configurable by the user. + +14.2 Accept-Charset + + The Accept-Charset request-header field can be used to indicate what + character sets are acceptable for the response. This field allows + clients capable of understanding more comprehensive or special- + purpose character sets to signal that capability to a server which is + capable of representing documents in those character sets. + + Accept-Charset = "Accept-Charset" ":" + 1#( ( charset | "*" )[ ";" "q" "=" qvalue ] ) + + + Character set values are described in section 3.4. Each charset MAY + be given an associated quality value which represents the user's + preference for that charset. The default value is q=1. An example is + + Accept-Charset: iso-8859-5, unicode-1-1;q=0.8 + + The special value "*", if present in the Accept-Charset field, + matches every character set (including ISO-8859-1) which is not + mentioned elsewhere in the Accept-Charset field. If no "*" is present + in an Accept-Charset field, then all character sets not explicitly + mentioned get a quality value of 0, except for ISO-8859-1, which gets + a quality value of 1 if not explicitly mentioned. + + If no Accept-Charset header is present, the default is that any + character set is acceptable. If an Accept-Charset header is present, + and if the server cannot send a response which is acceptable + according to the Accept-Charset header, then the server SHOULD send + an error response with the 406 (not acceptable) status code, though + the sending of an unacceptable response is also allowed. + +14.3 Accept-Encoding + + The Accept-Encoding request-header field is similar to Accept, but + restricts the content-codings (section 3.5) that are acceptable in + the response. + + Accept-Encoding = "Accept-Encoding" ":" + + + +Fielding, et al. Standards Track [Page 102] + +RFC 2616 HTTP/1.1 June 1999 + + + 1#( codings [ ";" "q" "=" qvalue ] ) + codings = ( content-coding | "*" ) + + Examples of its use are: + + Accept-Encoding: compress, gzip + Accept-Encoding: + Accept-Encoding: * + Accept-Encoding: compress;q=0.5, gzip;q=1.0 + Accept-Encoding: gzip;q=1.0, identity; q=0.5, *;q=0 + + A server tests whether a content-coding is acceptable, according to + an Accept-Encoding field, using these rules: + + 1. If the content-coding is one of the content-codings listed in + the Accept-Encoding field, then it is acceptable, unless it is + accompanied by a qvalue of 0. (As defined in section 3.9, a + qvalue of 0 means "not acceptable.") + + 2. The special "*" symbol in an Accept-Encoding field matches any + available content-coding not explicitly listed in the header + field. + + 3. If multiple content-codings are acceptable, then the acceptable + content-coding with the highest non-zero qvalue is preferred. + + 4. The "identity" content-coding is always acceptable, unless + specifically refused because the Accept-Encoding field includes + "identity;q=0", or because the field includes "*;q=0" and does + not explicitly include the "identity" content-coding. If the + Accept-Encoding field-value is empty, then only the "identity" + encoding is acceptable. + + If an Accept-Encoding field is present in a request, and if the + server cannot send a response which is acceptable according to the + Accept-Encoding header, then the server SHOULD send an error response + with the 406 (Not Acceptable) status code. + + If no Accept-Encoding field is present in a request, the server MAY + assume that the client will accept any content coding. In this case, + if "identity" is one of the available content-codings, then the + server SHOULD use the "identity" content-coding, unless it has + additional information that a different content-coding is meaningful + to the client. + + Note: If the request does not include an Accept-Encoding field, + and if the "identity" content-coding is unavailable, then + content-codings commonly understood by HTTP/1.0 clients (i.e., + + + +Fielding, et al. Standards Track [Page 103] + +RFC 2616 HTTP/1.1 June 1999 + + + "gzip" and "compress") are preferred; some older clients + improperly display messages sent with other content-codings. The + server might also make this decision based on information about + the particular user-agent or client. + + Note: Most HTTP/1.0 applications do not recognize or obey qvalues + associated with content-codings. This means that qvalues will not + work and are not permitted with x-gzip or x-compress. + +14.4 Accept-Language + + The Accept-Language request-header field is similar to Accept, but + restricts the set of natural languages that are preferred as a + response to the request. Language tags are defined in section 3.10. + + Accept-Language = "Accept-Language" ":" + 1#( language-range [ ";" "q" "=" qvalue ] ) + language-range = ( ( 1*8ALPHA *( "-" 1*8ALPHA ) ) | "*" ) + + Each language-range MAY be given an associated quality value which + represents an estimate of the user's preference for the languages + specified by that range. The quality value defaults to "q=1". For + example, + + Accept-Language: da, en-gb;q=0.8, en;q=0.7 + + would mean: "I prefer Danish, but will accept British English and + other types of English." A language-range matches a language-tag if + it exactly equals the tag, or if it exactly equals a prefix of the + tag such that the first tag character following the prefix is "-". + The special range "*", if present in the Accept-Language field, + matches every tag not matched by any other range present in the + Accept-Language field. + + Note: This use of a prefix matching rule does not imply that + language tags are assigned to languages in such a way that it is + always true that if a user understands a language with a certain + tag, then this user will also understand all languages with tags + for which this tag is a prefix. The prefix rule simply allows the + use of prefix tags if this is the case. + + The language quality factor assigned to a language-tag by the + Accept-Language field is the quality value of the longest language- + range in the field that matches the language-tag. If no language- + range in the field matches the tag, the language quality factor + assigned is 0. If no Accept-Language header is present in the + request, the server + + + + +Fielding, et al. Standards Track [Page 104] + +RFC 2616 HTTP/1.1 June 1999 + + + SHOULD assume that all languages are equally acceptable. If an + Accept-Language header is present, then all languages which are + assigned a quality factor greater than 0 are acceptable. + + It might be contrary to the privacy expectations of the user to send + an Accept-Language header with the complete linguistic preferences of + the user in every request. For a discussion of this issue, see + section 15.1.4. + + As intelligibility is highly dependent on the individual user, it is + recommended that client applications make the choice of linguistic + preference available to the user. If the choice is not made + available, then the Accept-Language header field MUST NOT be given in + the request. + + Note: When making the choice of linguistic preference available to + the user, we remind implementors of the fact that users are not + familiar with the details of language matching as described above, + and should provide appropriate guidance. As an example, users + might assume that on selecting "en-gb", they will be served any + kind of English document if British English is not available. A + user agent might suggest in such a case to add "en" to get the + best matching behavior. + +14.5 Accept-Ranges + + The Accept-Ranges response-header field allows the server to + indicate its acceptance of range requests for a resource: + + Accept-Ranges = "Accept-Ranges" ":" acceptable-ranges + acceptable-ranges = 1#range-unit | "none" + + Origin servers that accept byte-range requests MAY send + + Accept-Ranges: bytes + + but are not required to do so. Clients MAY generate byte-range + requests without having received this header for the resource + involved. Range units are defined in section 3.12. + + Servers that do not accept any kind of range request for a + resource MAY send + + Accept-Ranges: none + + to advise the client not to attempt a range request. + + + + + +Fielding, et al. Standards Track [Page 105] + +RFC 2616 HTTP/1.1 June 1999 + + +14.6 Age + + The Age response-header field conveys the sender's estimate of the + amount of time since the response (or its revalidation) was + generated at the origin server. A cached response is "fresh" if + its age does not exceed its freshness lifetime. Age values are + calculated as specified in section 13.2.3. + + Age = "Age" ":" age-value + age-value = delta-seconds + + Age values are non-negative decimal integers, representing time in + seconds. + + If a cache receives a value larger than the largest positive + integer it can represent, or if any of its age calculations + overflows, it MUST transmit an Age header with a value of + 2147483648 (2^31). An HTTP/1.1 server that includes a cache MUST + include an Age header field in every response generated from its + own cache. Caches SHOULD use an arithmetic type of at least 31 + bits of range. + +14.7 Allow + + The Allow entity-header field lists the set of methods supported + by the resource identified by the Request-URI. The purpose of this + field is strictly to inform the recipient of valid methods + associated with the resource. An Allow header field MUST be + present in a 405 (Method Not Allowed) response. + + Allow = "Allow" ":" #Method + + Example of use: + + Allow: GET, HEAD, PUT + + This field cannot prevent a client from trying other methods. + However, the indications given by the Allow header field value + SHOULD be followed. The actual set of allowed methods is defined + by the origin server at the time of each request. + + The Allow header field MAY be provided with a PUT request to + recommend the methods to be supported by the new or modified + resource. The server is not required to support these methods and + SHOULD include an Allow header in the response giving the actual + supported methods. + + + + + +Fielding, et al. Standards Track [Page 106] + +RFC 2616 HTTP/1.1 June 1999 + + + A proxy MUST NOT modify the Allow header field even if it does not + understand all the methods specified, since the user agent might + have other means of communicating with the origin server. + +14.8 Authorization + + A user agent that wishes to authenticate itself with a server-- + usually, but not necessarily, after receiving a 401 response--does + so by including an Authorization request-header field with the + request. The Authorization field value consists of credentials + containing the authentication information of the user agent for + the realm of the resource being requested. + + Authorization = "Authorization" ":" credentials + + HTTP access authentication is described in "HTTP Authentication: + Basic and Digest Access Authentication" [43]. If a request is + authenticated and a realm specified, the same credentials SHOULD + be valid for all other requests within this realm (assuming that + the authentication scheme itself does not require otherwise, such + as credentials that vary according to a challenge value or using + synchronized clocks). + + When a shared cache (see section 13.7) receives a request + containing an Authorization field, it MUST NOT return the + corresponding response as a reply to any other request, unless one + of the following specific exceptions holds: + + 1. If the response includes the "s-maxage" cache-control + directive, the cache MAY use that response in replying to a + subsequent request. But (if the specified maximum age has + passed) a proxy cache MUST first revalidate it with the origin + server, using the request-headers from the new request to allow + the origin server to authenticate the new request. (This is the + defined behavior for s-maxage.) If the response includes "s- + maxage=0", the proxy MUST always revalidate it before re-using + it. + + 2. If the response includes the "must-revalidate" cache-control + directive, the cache MAY use that response in replying to a + subsequent request. But if the response is stale, all caches + MUST first revalidate it with the origin server, using the + request-headers from the new request to allow the origin server + to authenticate the new request. + + 3. If the response includes the "public" cache-control directive, + it MAY be returned in reply to any subsequent request. + + + + +Fielding, et al. Standards Track [Page 107] + +RFC 2616 HTTP/1.1 June 1999 + + +14.9 Cache-Control + + The Cache-Control general-header field is used to specify directives + that MUST be obeyed by all caching mechanisms along the + request/response chain. The directives specify behavior intended to + prevent caches from adversely interfering with the request or + response. These directives typically override the default caching + algorithms. Cache directives are unidirectional in that the presence + of a directive in a request does not imply that the same directive is + to be given in the response. + + Note that HTTP/1.0 caches might not implement Cache-Control and + might only implement Pragma: no-cache (see section 14.32). + + Cache directives MUST be passed through by a proxy or gateway + application, regardless of their significance to that application, + since the directives might be applicable to all recipients along the + request/response chain. It is not possible to specify a cache- + directive for a specific cache. + + Cache-Control = "Cache-Control" ":" 1#cache-directive + + cache-directive = cache-request-directive + | cache-response-directive + + cache-request-directive = + "no-cache" ; Section 14.9.1 + | "no-store" ; Section 14.9.2 + | "max-age" "=" delta-seconds ; Section 14.9.3, 14.9.4 + | "max-stale" [ "=" delta-seconds ] ; Section 14.9.3 + | "min-fresh" "=" delta-seconds ; Section 14.9.3 + | "no-transform" ; Section 14.9.5 + | "only-if-cached" ; Section 14.9.4 + | cache-extension ; Section 14.9.6 + + cache-response-directive = + "public" ; Section 14.9.1 + | "private" [ "=" <"> 1#field-name <"> ] ; Section 14.9.1 + | "no-cache" [ "=" <"> 1#field-name <"> ]; Section 14.9.1 + | "no-store" ; Section 14.9.2 + | "no-transform" ; Section 14.9.5 + | "must-revalidate" ; Section 14.9.4 + | "proxy-revalidate" ; Section 14.9.4 + | "max-age" "=" delta-seconds ; Section 14.9.3 + | "s-maxage" "=" delta-seconds ; Section 14.9.3 + | cache-extension ; Section 14.9.6 + + cache-extension = token [ "=" ( token | quoted-string ) ] + + + +Fielding, et al. Standards Track [Page 108] + +RFC 2616 HTTP/1.1 June 1999 + + + When a directive appears without any 1#field-name parameter, the + directive applies to the entire request or response. When such a + directive appears with a 1#field-name parameter, it applies only to + the named field or fields, and not to the rest of the request or + response. This mechanism supports extensibility; implementations of + future versions of the HTTP protocol might apply these directives to + header fields not defined in HTTP/1.1. + + The cache-control directives can be broken down into these general + categories: + + - Restrictions on what are cacheable; these may only be imposed by + the origin server. + + - Restrictions on what may be stored by a cache; these may be + imposed by either the origin server or the user agent. + + - Modifications of the basic expiration mechanism; these may be + imposed by either the origin server or the user agent. + + - Controls over cache revalidation and reload; these may only be + imposed by a user agent. + + - Control over transformation of entities. + + - Extensions to the caching system. + +14.9.1 What is Cacheable + + By default, a response is cacheable if the requirements of the + request method, request header fields, and the response status + indicate that it is cacheable. Section 13.4 summarizes these defaults + for cacheability. The following Cache-Control response directives + allow an origin server to override the default cacheability of a + response: + + public + Indicates that the response MAY be cached by any cache, even if it + would normally be non-cacheable or cacheable only within a non- + shared cache. (See also Authorization, section 14.8, for + additional details.) + + private + Indicates that all or part of the response message is intended for + a single user and MUST NOT be cached by a shared cache. This + allows an origin server to state that the specified parts of the + + + + + +Fielding, et al. Standards Track [Page 109] + +RFC 2616 HTTP/1.1 June 1999 + + + response are intended for only one user and are not a valid + response for requests by other users. A private (non-shared) cache + MAY cache the response. + + Note: This usage of the word private only controls where the + response may be cached, and cannot ensure the privacy of the + message content. + + no-cache + If the no-cache directive does not specify a field-name, then a + cache MUST NOT use the response to satisfy a subsequent request + without successful revalidation with the origin server. This + allows an origin server to prevent caching even by caches that + have been configured to return stale responses to client requests. + + If the no-cache directive does specify one or more field-names, + then a cache MAY use the response to satisfy a subsequent request, + subject to any other restrictions on caching. However, the + specified field-name(s) MUST NOT be sent in the response to a + subsequent request without successful revalidation with the origin + server. This allows an origin server to prevent the re-use of + certain header fields in a response, while still allowing caching + of the rest of the response. + + Note: Most HTTP/1.0 caches will not recognize or obey this + directive. + +14.9.2 What May be Stored by Caches + + no-store + The purpose of the no-store directive is to prevent the + inadvertent release or retention of sensitive information (for + example, on backup tapes). The no-store directive applies to the + entire message, and MAY be sent either in a response or in a + request. If sent in a request, a cache MUST NOT store any part of + either this request or any response to it. If sent in a response, + a cache MUST NOT store any part of either this response or the + request that elicited it. This directive applies to both non- + shared and shared caches. "MUST NOT store" in this context means + that the cache MUST NOT intentionally store the information in + non-volatile storage, and MUST make a best-effort attempt to + remove the information from volatile storage as promptly as + possible after forwarding it. + + Even when this directive is associated with a response, users + might explicitly store such a response outside of the caching + system (e.g., with a "Save As" dialog). History buffers MAY store + such responses as part of their normal operation. + + + +Fielding, et al. Standards Track [Page 110] + +RFC 2616 HTTP/1.1 June 1999 + + + The purpose of this directive is to meet the stated requirements + of certain users and service authors who are concerned about + accidental releases of information via unanticipated accesses to + cache data structures. While the use of this directive might + improve privacy in some cases, we caution that it is NOT in any + way a reliable or sufficient mechanism for ensuring privacy. In + particular, malicious or compromised caches might not recognize or + obey this directive, and communications networks might be + vulnerable to eavesdropping. + +14.9.3 Modifications of the Basic Expiration Mechanism + + The expiration time of an entity MAY be specified by the origin + server using the Expires header (see section 14.21). Alternatively, + it MAY be specified using the max-age directive in a response. When + the max-age cache-control directive is present in a cached response, + the response is stale if its current age is greater than the age + value given (in seconds) at the time of a new request for that + resource. The max-age directive on a response implies that the + response is cacheable (i.e., "public") unless some other, more + restrictive cache directive is also present. + + If a response includes both an Expires header and a max-age + directive, the max-age directive overrides the Expires header, even + if the Expires header is more restrictive. This rule allows an origin + server to provide, for a given response, a longer expiration time to + an HTTP/1.1 (or later) cache than to an HTTP/1.0 cache. This might be + useful if certain HTTP/1.0 caches improperly calculate ages or + expiration times, perhaps due to desynchronized clocks. + + Many HTTP/1.0 cache implementations will treat an Expires value that + is less than or equal to the response Date value as being equivalent + to the Cache-Control response directive "no-cache". If an HTTP/1.1 + cache receives such a response, and the response does not include a + Cache-Control header field, it SHOULD consider the response to be + non-cacheable in order to retain compatibility with HTTP/1.0 servers. + + Note: An origin server might wish to use a relatively new HTTP + cache control feature, such as the "private" directive, on a + network including older caches that do not understand that + feature. The origin server will need to combine the new feature + with an Expires field whose value is less than or equal to the + Date value. This will prevent older caches from improperly + caching the response. + + + + + + + +Fielding, et al. Standards Track [Page 111] + +RFC 2616 HTTP/1.1 June 1999 + + + s-maxage + If a response includes an s-maxage directive, then for a shared + cache (but not for a private cache), the maximum age specified by + this directive overrides the maximum age specified by either the + max-age directive or the Expires header. The s-maxage directive + also implies the semantics of the proxy-revalidate directive (see + section 14.9.4), i.e., that the shared cache must not use the + entry after it becomes stale to respond to a subsequent request + without first revalidating it with the origin server. The s- + maxage directive is always ignored by a private cache. + + Note that most older caches, not compliant with this specification, + do not implement any cache-control directives. An origin server + wishing to use a cache-control directive that restricts, but does not + prevent, caching by an HTTP/1.1-compliant cache MAY exploit the + requirement that the max-age directive overrides the Expires header, + and the fact that pre-HTTP/1.1-compliant caches do not observe the + max-age directive. + + Other directives allow a user agent to modify the basic expiration + mechanism. These directives MAY be specified on a request: + + max-age + Indicates that the client is willing to accept a response whose + age is no greater than the specified time in seconds. Unless max- + stale directive is also included, the client is not willing to + accept a stale response. + + min-fresh + Indicates that the client is willing to accept a response whose + freshness lifetime is no less than its current age plus the + specified time in seconds. That is, the client wants a response + that will still be fresh for at least the specified number of + seconds. + + max-stale + Indicates that the client is willing to accept a response that has + exceeded its expiration time. If max-stale is assigned a value, + then the client is willing to accept a response that has exceeded + its expiration time by no more than the specified number of + seconds. If no value is assigned to max-stale, then the client is + willing to accept a stale response of any age. + + If a cache returns a stale response, either because of a max-stale + directive on a request, or because the cache is configured to + override the expiration time of a response, the cache MUST attach a + Warning header to the stale response, using Warning 110 (Response is + stale). + + + +Fielding, et al. Standards Track [Page 112] + +RFC 2616 HTTP/1.1 June 1999 + + + A cache MAY be configured to return stale responses without + validation, but only if this does not conflict with any "MUST"-level + requirements concerning cache validation (e.g., a "must-revalidate" + cache-control directive). + + If both the new request and the cached entry include "max-age" + directives, then the lesser of the two values is used for determining + the freshness of the cached entry for that request. + +14.9.4 Cache Revalidation and Reload Controls + + Sometimes a user agent might want or need to insist that a cache + revalidate its cache entry with the origin server (and not just with + the next cache along the path to the origin server), or to reload its + cache entry from the origin server. End-to-end revalidation might be + necessary if either the cache or the origin server has overestimated + the expiration time of the cached response. End-to-end reload may be + necessary if the cache entry has become corrupted for some reason. + + End-to-end revalidation may be requested either when the client does + not have its own local cached copy, in which case we call it + "unspecified end-to-end revalidation", or when the client does have a + local cached copy, in which case we call it "specific end-to-end + revalidation." + + The client can specify these three kinds of action using Cache- + Control request directives: + + End-to-end reload + The request includes a "no-cache" cache-control directive or, for + compatibility with HTTP/1.0 clients, "Pragma: no-cache". Field + names MUST NOT be included with the no-cache directive in a + request. The server MUST NOT use a cached copy when responding to + such a request. + + Specific end-to-end revalidation + The request includes a "max-age=0" cache-control directive, which + forces each cache along the path to the origin server to + revalidate its own entry, if any, with the next cache or server. + The initial request includes a cache-validating conditional with + the client's current validator. + + Unspecified end-to-end revalidation + The request includes "max-age=0" cache-control directive, which + forces each cache along the path to the origin server to + revalidate its own entry, if any, with the next cache or server. + The initial request does not include a cache-validating + + + + +Fielding, et al. Standards Track [Page 113] + +RFC 2616 HTTP/1.1 June 1999 + + + conditional; the first cache along the path (if any) that holds a + cache entry for this resource includes a cache-validating + conditional with its current validator. + + max-age + When an intermediate cache is forced, by means of a max-age=0 + directive, to revalidate its own cache entry, and the client has + supplied its own validator in the request, the supplied validator + might differ from the validator currently stored with the cache + entry. In this case, the cache MAY use either validator in making + its own request without affecting semantic transparency. + + However, the choice of validator might affect performance. The + best approach is for the intermediate cache to use its own + validator when making its request. If the server replies with 304 + (Not Modified), then the cache can return its now validated copy + to the client with a 200 (OK) response. If the server replies with + a new entity and cache validator, however, the intermediate cache + can compare the returned validator with the one provided in the + client's request, using the strong comparison function. If the + client's validator is equal to the origin server's, then the + intermediate cache simply returns 304 (Not Modified). Otherwise, + it returns the new entity with a 200 (OK) response. + + If a request includes the no-cache directive, it SHOULD NOT + include min-fresh, max-stale, or max-age. + + only-if-cached + In some cases, such as times of extremely poor network + connectivity, a client may want a cache to return only those + responses that it currently has stored, and not to reload or + revalidate with the origin server. To do this, the client may + include the only-if-cached directive in a request. If it receives + this directive, a cache SHOULD either respond using a cached entry + that is consistent with the other constraints of the request, or + respond with a 504 (Gateway Timeout) status. However, if a group + of caches is being operated as a unified system with good internal + connectivity, such a request MAY be forwarded within that group of + caches. + + must-revalidate + Because a cache MAY be configured to ignore a server's specified + expiration time, and because a client request MAY include a max- + stale directive (which has a similar effect), the protocol also + includes a mechanism for the origin server to require revalidation + of a cache entry on any subsequent use. When the must-revalidate + directive is present in a response received by a cache, that cache + MUST NOT use the entry after it becomes stale to respond to a + + + +Fielding, et al. Standards Track [Page 114] + +RFC 2616 HTTP/1.1 June 1999 + + + subsequent request without first revalidating it with the origin + server. (I.e., the cache MUST do an end-to-end revalidation every + time, if, based solely on the origin server's Expires or max-age + value, the cached response is stale.) + + The must-revalidate directive is necessary to support reliable + operation for certain protocol features. In all circumstances an + HTTP/1.1 cache MUST obey the must-revalidate directive; in + particular, if the cache cannot reach the origin server for any + reason, it MUST generate a 504 (Gateway Timeout) response. + + Servers SHOULD send the must-revalidate directive if and only if + failure to revalidate a request on the entity could result in + incorrect operation, such as a silently unexecuted financial + transaction. Recipients MUST NOT take any automated action that + violates this directive, and MUST NOT automatically provide an + unvalidated copy of the entity if revalidation fails. + + Although this is not recommended, user agents operating under + severe connectivity constraints MAY violate this directive but, if + so, MUST explicitly warn the user that an unvalidated response has + been provided. The warning MUST be provided on each unvalidated + access, and SHOULD require explicit user confirmation. + + proxy-revalidate + The proxy-revalidate directive has the same meaning as the must- + revalidate directive, except that it does not apply to non-shared + user agent caches. It can be used on a response to an + authenticated request to permit the user's cache to store and + later return the response without needing to revalidate it (since + it has already been authenticated once by that user), while still + requiring proxies that service many users to revalidate each time + (in order to make sure that each user has been authenticated). + Note that such authenticated responses also need the public cache + control directive in order to allow them to be cached at all. + +14.9.5 No-Transform Directive + + no-transform + Implementors of intermediate caches (proxies) have found it useful + to convert the media type of certain entity bodies. A non- + transparent proxy might, for example, convert between image + formats in order to save cache space or to reduce the amount of + traffic on a slow link. + + Serious operational problems occur, however, when these + transformations are applied to entity bodies intended for certain + kinds of applications. For example, applications for medical + + + +Fielding, et al. Standards Track [Page 115] + +RFC 2616 HTTP/1.1 June 1999 + + + imaging, scientific data analysis and those using end-to-end + authentication, all depend on receiving an entity body that is bit + for bit identical to the original entity-body. + + Therefore, if a message includes the no-transform directive, an + intermediate cache or proxy MUST NOT change those headers that are + listed in section 13.5.2 as being subject to the no-transform + directive. This implies that the cache or proxy MUST NOT change + any aspect of the entity-body that is specified by these headers, + including the value of the entity-body itself. + +14.9.6 Cache Control Extensions + + The Cache-Control header field can be extended through the use of one + or more cache-extension tokens, each with an optional assigned value. + Informational extensions (those which do not require a change in + cache behavior) MAY be added without changing the semantics of other + directives. Behavioral extensions are designed to work by acting as + modifiers to the existing base of cache directives. Both the new + directive and the standard directive are supplied, such that + applications which do not understand the new directive will default + to the behavior specified by the standard directive, and those that + understand the new directive will recognize it as modifying the + requirements associated with the standard directive. In this way, + extensions to the cache-control directives can be made without + requiring changes to the base protocol. + + This extension mechanism depends on an HTTP cache obeying all of the + cache-control directives defined for its native HTTP-version, obeying + certain extensions, and ignoring all directives that it does not + understand. + + For example, consider a hypothetical new response directive called + community which acts as a modifier to the private directive. We + define this new directive to mean that, in addition to any non-shared + cache, any cache which is shared only by members of the community + named within its value may cache the response. An origin server + wishing to allow the UCI community to use an otherwise private + response in their shared cache(s) could do so by including + + Cache-Control: private, community="UCI" + + A cache seeing this header field will act correctly even if the cache + does not understand the community cache-extension, since it will also + see and understand the private directive and thus default to the safe + behavior. + + + + + +Fielding, et al. Standards Track [Page 116] + +RFC 2616 HTTP/1.1 June 1999 + + + Unrecognized cache-directives MUST be ignored; it is assumed that any + cache-directive likely to be unrecognized by an HTTP/1.1 cache will + be combined with standard directives (or the response's default + cacheability) such that the cache behavior will remain minimally + correct even if the cache does not understand the extension(s). + +14.10 Connection + + The Connection general-header field allows the sender to specify + options that are desired for that particular connection and MUST NOT + be communicated by proxies over further connections. + + The Connection header has the following grammar: + + Connection = "Connection" ":" 1#(connection-token) + connection-token = token + + HTTP/1.1 proxies MUST parse the Connection header field before a + message is forwarded and, for each connection-token in this field, + remove any header field(s) from the message with the same name as the + connection-token. Connection options are signaled by the presence of + a connection-token in the Connection header field, not by any + corresponding additional header field(s), since the additional header + field may not be sent if there are no parameters associated with that + connection option. + + Message headers listed in the Connection header MUST NOT include + end-to-end headers, such as Cache-Control. + + HTTP/1.1 defines the "close" connection option for the sender to + signal that the connection will be closed after completion of the + response. For example, + + Connection: close + + in either the request or the response header fields indicates that + the connection SHOULD NOT be considered `persistent' (section 8.1) + after the current request/response is complete. + + HTTP/1.1 applications that do not support persistent connections MUST + include the "close" connection option in every message. + + A system receiving an HTTP/1.0 (or lower-version) message that + includes a Connection header MUST, for each connection-token in this + field, remove and ignore any header field(s) from the message with + the same name as the connection-token. This protects against mistaken + forwarding of such header fields by pre-HTTP/1.1 proxies. See section + 19.6.2. + + + +Fielding, et al. Standards Track [Page 117] + +RFC 2616 HTTP/1.1 June 1999 + + +14.11 Content-Encoding + + The Content-Encoding entity-header field is used as a modifier to the + media-type. When present, its value indicates what additional content + codings have been applied to the entity-body, and thus what decoding + mechanisms must be applied in order to obtain the media-type + referenced by the Content-Type header field. Content-Encoding is + primarily used to allow a document to be compressed without losing + the identity of its underlying media type. + + Content-Encoding = "Content-Encoding" ":" 1#content-coding + + Content codings are defined in section 3.5. An example of its use is + + Content-Encoding: gzip + + The content-coding is a characteristic of the entity identified by + the Request-URI. Typically, the entity-body is stored with this + encoding and is only decoded before rendering or analogous usage. + However, a non-transparent proxy MAY modify the content-coding if the + new coding is known to be acceptable to the recipient, unless the + "no-transform" cache-control directive is present in the message. + + If the content-coding of an entity is not "identity", then the + response MUST include a Content-Encoding entity-header (section + 14.11) that lists the non-identity content-coding(s) used. + + If the content-coding of an entity in a request message is not + acceptable to the origin server, the server SHOULD respond with a + status code of 415 (Unsupported Media Type). + + If multiple encodings have been applied to an entity, the content + codings MUST be listed in the order in which they were applied. + Additional information about the encoding parameters MAY be provided + by other entity-header fields not defined by this specification. + +14.12 Content-Language + + The Content-Language entity-header field describes the natural + language(s) of the intended audience for the enclosed entity. Note + that this might not be equivalent to all the languages used within + the entity-body. + + Content-Language = "Content-Language" ":" 1#language-tag + + + + + + + +Fielding, et al. Standards Track [Page 118] + +RFC 2616 HTTP/1.1 June 1999 + + + Language tags are defined in section 3.10. The primary purpose of + Content-Language is to allow a user to identify and differentiate + entities according to the user's own preferred language. Thus, if the + body content is intended only for a Danish-literate audience, the + appropriate field is + + Content-Language: da + + If no Content-Language is specified, the default is that the content + is intended for all language audiences. This might mean that the + sender does not consider it to be specific to any natural language, + or that the sender does not know for which language it is intended. + + Multiple languages MAY be listed for content that is intended for + multiple audiences. For example, a rendition of the "Treaty of + Waitangi," presented simultaneously in the original Maori and English + versions, would call for + + Content-Language: mi, en + + However, just because multiple languages are present within an entity + does not mean that it is intended for multiple linguistic audiences. + An example would be a beginner's language primer, such as "A First + Lesson in Latin," which is clearly intended to be used by an + English-literate audience. In this case, the Content-Language would + properly only include "en". + + Content-Language MAY be applied to any media type -- it is not + limited to textual documents. + +14.13 Content-Length + + The Content-Length entity-header field indicates the size of the + entity-body, in decimal number of OCTETs, sent to the recipient or, + in the case of the HEAD method, the size of the entity-body that + would have been sent had the request been a GET. + + Content-Length = "Content-Length" ":" 1*DIGIT + + An example is + + Content-Length: 3495 + + Applications SHOULD use this field to indicate the transfer-length of + the message-body, unless this is prohibited by the rules in section + 4.4. + + + + + +Fielding, et al. Standards Track [Page 119] + +RFC 2616 HTTP/1.1 June 1999 + + + Any Content-Length greater than or equal to zero is a valid value. + Section 4.4 describes how to determine the length of a message-body + if a Content-Length is not given. + + Note that the meaning of this field is significantly different from + the corresponding definition in MIME, where it is an optional field + used within the "message/external-body" content-type. In HTTP, it + SHOULD be sent whenever the message's length can be determined prior + to being transferred, unless this is prohibited by the rules in + section 4.4. + +14.14 Content-Location + + The Content-Location entity-header field MAY be used to supply the + resource location for the entity enclosed in the message when that + entity is accessible from a location separate from the requested + resource's URI. A server SHOULD provide a Content-Location for the + variant corresponding to the response entity; especially in the case + where a resource has multiple entities associated with it, and those + entities actually have separate locations by which they might be + individually accessed, the server SHOULD provide a Content-Location + for the particular variant which is returned. + + Content-Location = "Content-Location" ":" + ( absoluteURI | relativeURI ) + + The value of Content-Location also defines the base URI for the + entity. + + The Content-Location value is not a replacement for the original + requested URI; it is only a statement of the location of the resource + corresponding to this particular entity at the time of the request. + Future requests MAY specify the Content-Location URI as the request- + URI if the desire is to identify the source of that particular + entity. + + A cache cannot assume that an entity with a Content-Location + different from the URI used to retrieve it can be used to respond to + later requests on that Content-Location URI. However, the Content- + Location can be used to differentiate between multiple entities + retrieved from a single requested resource, as described in section + 13.6. + + If the Content-Location is a relative URI, the relative URI is + interpreted relative to the Request-URI. + + The meaning of the Content-Location header in PUT or POST requests is + undefined; servers are free to ignore it in those cases. + + + +Fielding, et al. Standards Track [Page 120] + +RFC 2616 HTTP/1.1 June 1999 + + +14.15 Content-MD5 + + The Content-MD5 entity-header field, as defined in RFC 1864 [23], is + an MD5 digest of the entity-body for the purpose of providing an + end-to-end message integrity check (MIC) of the entity-body. (Note: a + MIC is good for detecting accidental modification of the entity-body + in transit, but is not proof against malicious attacks.) + + Content-MD5 = "Content-MD5" ":" md5-digest + md5-digest = + + The Content-MD5 header field MAY be generated by an origin server or + client to function as an integrity check of the entity-body. Only + origin servers or clients MAY generate the Content-MD5 header field; + proxies and gateways MUST NOT generate it, as this would defeat its + value as an end-to-end integrity check. Any recipient of the entity- + body, including gateways and proxies, MAY check that the digest value + in this header field matches that of the entity-body as received. + + The MD5 digest is computed based on the content of the entity-body, + including any content-coding that has been applied, but not including + any transfer-encoding applied to the message-body. If the message is + received with a transfer-encoding, that encoding MUST be removed + prior to checking the Content-MD5 value against the received entity. + + This has the result that the digest is computed on the octets of the + entity-body exactly as, and in the order that, they would be sent if + no transfer-encoding were being applied. + + HTTP extends RFC 1864 to permit the digest to be computed for MIME + composite media-types (e.g., multipart/* and message/rfc822), but + this does not change how the digest is computed as defined in the + preceding paragraph. + + There are several consequences of this. The entity-body for composite + types MAY contain many body-parts, each with its own MIME and HTTP + headers (including Content-MD5, Content-Transfer-Encoding, and + Content-Encoding headers). If a body-part has a Content-Transfer- + Encoding or Content-Encoding header, it is assumed that the content + of the body-part has had the encoding applied, and the body-part is + included in the Content-MD5 digest as is -- i.e., after the + application. The Transfer-Encoding header field is not allowed within + body-parts. + + Conversion of all line breaks to CRLF MUST NOT be done before + computing or checking the digest: the line break convention used in + the text actually transmitted MUST be left unaltered when computing + the digest. + + + +Fielding, et al. Standards Track [Page 121] + +RFC 2616 HTTP/1.1 June 1999 + + + Note: while the definition of Content-MD5 is exactly the same for + HTTP as in RFC 1864 for MIME entity-bodies, there are several ways + in which the application of Content-MD5 to HTTP entity-bodies + differs from its application to MIME entity-bodies. One is that + HTTP, unlike MIME, does not use Content-Transfer-Encoding, and + does use Transfer-Encoding and Content-Encoding. Another is that + HTTP more frequently uses binary content types than MIME, so it is + worth noting that, in such cases, the byte order used to compute + the digest is the transmission byte order defined for the type. + Lastly, HTTP allows transmission of text types with any of several + line break conventions and not just the canonical form using CRLF. + +14.16 Content-Range + + The Content-Range entity-header is sent with a partial entity-body to + specify where in the full entity-body the partial body should be + applied. Range units are defined in section 3.12. + + Content-Range = "Content-Range" ":" content-range-spec + + content-range-spec = byte-content-range-spec + byte-content-range-spec = bytes-unit SP + byte-range-resp-spec "/" + ( instance-length | "*" ) + + byte-range-resp-spec = (first-byte-pos "-" last-byte-pos) + | "*" + instance-length = 1*DIGIT + + The header SHOULD indicate the total length of the full entity-body, + unless this length is unknown or difficult to determine. The asterisk + "*" character means that the instance-length is unknown at the time + when the response was generated. + + Unlike byte-ranges-specifier values (see section 14.35.1), a byte- + range-resp-spec MUST only specify one range, and MUST contain + absolute byte positions for both the first and last byte of the + range. + + A byte-content-range-spec with a byte-range-resp-spec whose last- + byte-pos value is less than its first-byte-pos value, or whose + instance-length value is less than or equal to its last-byte-pos + value, is invalid. The recipient of an invalid byte-content-range- + spec MUST ignore it and any content transferred along with it. + + A server sending a response with status code 416 (Requested range not + satisfiable) SHOULD include a Content-Range field with a byte-range- + resp-spec of "*". The instance-length specifies the current length of + + + +Fielding, et al. Standards Track [Page 122] + +RFC 2616 HTTP/1.1 June 1999 + + + the selected resource. A response with status code 206 (Partial + Content) MUST NOT include a Content-Range field with a byte-range- + resp-spec of "*". + + Examples of byte-content-range-spec values, assuming that the entity + contains a total of 1234 bytes: + + . The first 500 bytes: + bytes 0-499/1234 + + . The second 500 bytes: + bytes 500-999/1234 + + . All except for the first 500 bytes: + bytes 500-1233/1234 + + . The last 500 bytes: + bytes 734-1233/1234 + + When an HTTP message includes the content of a single range (for + example, a response to a request for a single range, or to a request + for a set of ranges that overlap without any holes), this content is + transmitted with a Content-Range header, and a Content-Length header + showing the number of bytes actually transferred. For example, + + HTTP/1.1 206 Partial content + Date: Wed, 15 Nov 1995 06:25:24 GMT + Last-Modified: Wed, 15 Nov 1995 04:58:08 GMT + Content-Range: bytes 21010-47021/47022 + Content-Length: 26012 + Content-Type: image/gif + + When an HTTP message includes the content of multiple ranges (for + example, a response to a request for multiple non-overlapping + ranges), these are transmitted as a multipart message. The multipart + media type used for this purpose is "multipart/byteranges" as defined + in appendix 19.2. See appendix 19.6.3 for a compatibility issue. + + A response to a request for a single range MUST NOT be sent using the + multipart/byteranges media type. A response to a request for + multiple ranges, whose result is a single range, MAY be sent as a + multipart/byteranges media type with one part. A client that cannot + decode a multipart/byteranges message MUST NOT ask for multiple + byte-ranges in a single request. + + When a client requests multiple byte-ranges in one request, the + server SHOULD return them in the order that they appeared in the + request. + + + +Fielding, et al. Standards Track [Page 123] + +RFC 2616 HTTP/1.1 June 1999 + + + If the server ignores a byte-range-spec because it is syntactically + invalid, the server SHOULD treat the request as if the invalid Range + header field did not exist. (Normally, this means return a 200 + response containing the full entity). + + If the server receives a request (other than one including an If- + Range request-header field) with an unsatisfiable Range request- + header field (that is, all of whose byte-range-spec values have a + first-byte-pos value greater than the current length of the selected + resource), it SHOULD return a response code of 416 (Requested range + not satisfiable) (section 10.4.17). + + Note: clients cannot depend on servers to send a 416 (Requested + range not satisfiable) response instead of a 200 (OK) response for + an unsatisfiable Range request-header, since not all servers + implement this request-header. + +14.17 Content-Type + + The Content-Type entity-header field indicates the media type of the + entity-body sent to the recipient or, in the case of the HEAD method, + the media type that would have been sent had the request been a GET. + + Content-Type = "Content-Type" ":" media-type + + Media types are defined in section 3.7. An example of the field is + + Content-Type: text/html; charset=ISO-8859-4 + + Further discussion of methods for identifying the media type of an + entity is provided in section 7.2.1. + +14.18 Date + + The Date general-header field represents the date and time at which + the message was originated, having the same semantics as orig-date in + RFC 822. The field value is an HTTP-date, as described in section + 3.3.1; it MUST be sent in RFC 1123 [8]-date format. + + Date = "Date" ":" HTTP-date + + An example is + + Date: Tue, 15 Nov 1994 08:12:31 GMT + + Origin servers MUST include a Date header field in all responses, + except in these cases: + + + + +Fielding, et al. Standards Track [Page 124] + +RFC 2616 HTTP/1.1 June 1999 + + + 1. If the response status code is 100 (Continue) or 101 (Switching + Protocols), the response MAY include a Date header field, at + the server's option. + + 2. If the response status code conveys a server error, e.g. 500 + (Internal Server Error) or 503 (Service Unavailable), and it is + inconvenient or impossible to generate a valid Date. + + 3. If the server does not have a clock that can provide a + reasonable approximation of the current time, its responses + MUST NOT include a Date header field. In this case, the rules + in section 14.18.1 MUST be followed. + + A received message that does not have a Date header field MUST be + assigned one by the recipient if the message will be cached by that + recipient or gatewayed via a protocol which requires a Date. An HTTP + implementation without a clock MUST NOT cache responses without + revalidating them on every use. An HTTP cache, especially a shared + cache, SHOULD use a mechanism, such as NTP [28], to synchronize its + clock with a reliable external standard. + + Clients SHOULD only send a Date header field in messages that include + an entity-body, as in the case of the PUT and POST requests, and even + then it is optional. A client without a clock MUST NOT send a Date + header field in a request. + + The HTTP-date sent in a Date header SHOULD NOT represent a date and + time subsequent to the generation of the message. It SHOULD represent + the best available approximation of the date and time of message + generation, unless the implementation has no means of generating a + reasonably accurate date and time. In theory, the date ought to + represent the moment just before the entity is generated. In + practice, the date can be generated at any time during the message + origination without affecting its semantic value. + +14.18.1 Clockless Origin Server Operation + + Some origin server implementations might not have a clock available. + An origin server without a clock MUST NOT assign Expires or Last- + Modified values to a response, unless these values were associated + with the resource by a system or user with a reliable clock. It MAY + assign an Expires value that is known, at or before server + configuration time, to be in the past (this allows "pre-expiration" + of responses without storing separate Expires values for each + resource). + + + + + + +Fielding, et al. Standards Track [Page 125] + +RFC 2616 HTTP/1.1 June 1999 + + +14.19 ETag + + The ETag response-header field provides the current value of the + entity tag for the requested variant. The headers used with entity + tags are described in sections 14.24, 14.26 and 14.44. The entity tag + MAY be used for comparison with other entities from the same resource + (see section 13.3.3). + + ETag = "ETag" ":" entity-tag + + Examples: + + ETag: "xyzzy" + ETag: W/"xyzzy" + ETag: "" + +14.20 Expect + + The Expect request-header field is used to indicate that particular + server behaviors are required by the client. + + Expect = "Expect" ":" 1#expectation + + expectation = "100-continue" | expectation-extension + expectation-extension = token [ "=" ( token | quoted-string ) + *expect-params ] + expect-params = ";" token [ "=" ( token | quoted-string ) ] + + + A server that does not understand or is unable to comply with any of + the expectation values in the Expect field of a request MUST respond + with appropriate error status. The server MUST respond with a 417 + (Expectation Failed) status if any of the expectations cannot be met + or, if there are other problems with the request, some other 4xx + status. + + This header field is defined with extensible syntax to allow for + future extensions. If a server receives a request containing an + Expect field that includes an expectation-extension that it does not + support, it MUST respond with a 417 (Expectation Failed) status. + + Comparison of expectation values is case-insensitive for unquoted + tokens (including the 100-continue token), and is case-sensitive for + quoted-string expectation-extensions. + + + + + + + +Fielding, et al. Standards Track [Page 126] + +RFC 2616 HTTP/1.1 June 1999 + + + The Expect mechanism is hop-by-hop: that is, an HTTP/1.1 proxy MUST + return a 417 (Expectation Failed) status if it receives a request + with an expectation that it cannot meet. However, the Expect + request-header itself is end-to-end; it MUST be forwarded if the + request is forwarded. + + Many older HTTP/1.0 and HTTP/1.1 applications do not understand the + Expect header. + + See section 8.2.3 for the use of the 100 (continue) status. + +14.21 Expires + + The Expires entity-header field gives the date/time after which the + response is considered stale. A stale cache entry may not normally be + returned by a cache (either a proxy cache or a user agent cache) + unless it is first validated with the origin server (or with an + intermediate cache that has a fresh copy of the entity). See section + 13.2 for further discussion of the expiration model. + + The presence of an Expires field does not imply that the original + resource will change or cease to exist at, before, or after that + time. + + The format is an absolute date and time as defined by HTTP-date in + section 3.3.1; it MUST be in RFC 1123 date format: + + Expires = "Expires" ":" HTTP-date + + An example of its use is + + Expires: Thu, 01 Dec 1994 16:00:00 GMT + + Note: if a response includes a Cache-Control field with the max- + age directive (see section 14.9.3), that directive overrides the + Expires field. + + HTTP/1.1 clients and caches MUST treat other invalid date formats, + especially including the value "0", as in the past (i.e., "already + expired"). + + To mark a response as "already expired," an origin server sends an + Expires date that is equal to the Date header value. (See the rules + for expiration calculations in section 13.2.4.) + + + + + + + +Fielding, et al. Standards Track [Page 127] + +RFC 2616 HTTP/1.1 June 1999 + + + To mark a response as "never expires," an origin server sends an + Expires date approximately one year from the time the response is + sent. HTTP/1.1 servers SHOULD NOT send Expires dates more than one + year in the future. + + The presence of an Expires header field with a date value of some + time in the future on a response that otherwise would by default be + non-cacheable indicates that the response is cacheable, unless + indicated otherwise by a Cache-Control header field (section 14.9). + +14.22 From + + The From request-header field, if given, SHOULD contain an Internet + e-mail address for the human user who controls the requesting user + agent. The address SHOULD be machine-usable, as defined by "mailbox" + in RFC 822 [9] as updated by RFC 1123 [8]: + + From = "From" ":" mailbox + + An example is: + + From: webmaster@w3.org + + This header field MAY be used for logging purposes and as a means for + identifying the source of invalid or unwanted requests. It SHOULD NOT + be used as an insecure form of access protection. The interpretation + of this field is that the request is being performed on behalf of the + person given, who accepts responsibility for the method performed. In + particular, robot agents SHOULD include this header so that the + person responsible for running the robot can be contacted if problems + occur on the receiving end. + + The Internet e-mail address in this field MAY be separate from the + Internet host which issued the request. For example, when a request + is passed through a proxy the original issuer's address SHOULD be + used. + + The client SHOULD NOT send the From header field without the user's + approval, as it might conflict with the user's privacy interests or + their site's security policy. It is strongly recommended that the + user be able to disable, enable, and modify the value of this field + at any time prior to a request. + +14.23 Host + + The Host request-header field specifies the Internet host and port + number of the resource being requested, as obtained from the original + URI given by the user or referring resource (generally an HTTP URL, + + + +Fielding, et al. Standards Track [Page 128] + +RFC 2616 HTTP/1.1 June 1999 + + + as described in section 3.2.2). The Host field value MUST represent + the naming authority of the origin server or gateway given by the + original URL. This allows the origin server or gateway to + differentiate between internally-ambiguous URLs, such as the root "/" + URL of a server for multiple host names on a single IP address. + + Host = "Host" ":" host [ ":" port ] ; Section 3.2.2 + + A "host" without any trailing port information implies the default + port for the service requested (e.g., "80" for an HTTP URL). For + example, a request on the origin server for + would properly include: + + GET /pub/WWW/ HTTP/1.1 + Host: www.w3.org + + A client MUST include a Host header field in all HTTP/1.1 request + messages . If the requested URI does not include an Internet host + name for the service being requested, then the Host header field MUST + be given with an empty value. An HTTP/1.1 proxy MUST ensure that any + request message it forwards does contain an appropriate Host header + field that identifies the service being requested by the proxy. All + Internet-based HTTP/1.1 servers MUST respond with a 400 (Bad Request) + status code to any HTTP/1.1 request message which lacks a Host header + field. + + See sections 5.2 and 19.6.1.1 for other requirements relating to + Host. + +14.24 If-Match + + The If-Match request-header field is used with a method to make it + conditional. A client that has one or more entities previously + obtained from the resource can verify that one of those entities is + current by including a list of their associated entity tags in the + If-Match header field. Entity tags are defined in section 3.11. The + purpose of this feature is to allow efficient updates of cached + information with a minimum amount of transaction overhead. It is also + used, on updating requests, to prevent inadvertent modification of + the wrong version of a resource. As a special case, the value "*" + matches any current entity of the resource. + + If-Match = "If-Match" ":" ( "*" | 1#entity-tag ) + + If any of the entity tags match the entity tag of the entity that + would have been returned in the response to a similar GET request + (without the If-Match header) on that resource, or if "*" is given + + + + +Fielding, et al. Standards Track [Page 129] + +RFC 2616 HTTP/1.1 June 1999 + + + and any current entity exists for that resource, then the server MAY + perform the requested method as if the If-Match header field did not + exist. + + A server MUST use the strong comparison function (see section 13.3.3) + to compare the entity tags in If-Match. + + If none of the entity tags match, or if "*" is given and no current + entity exists, the server MUST NOT perform the requested method, and + MUST return a 412 (Precondition Failed) response. This behavior is + most useful when the client wants to prevent an updating method, such + as PUT, from modifying a resource that has changed since the client + last retrieved it. + + If the request would, without the If-Match header field, result in + anything other than a 2xx or 412 status, then the If-Match header + MUST be ignored. + + The meaning of "If-Match: *" is that the method SHOULD be performed + if the representation selected by the origin server (or by a cache, + possibly using the Vary mechanism, see section 14.44) exists, and + MUST NOT be performed if the representation does not exist. + + A request intended to update a resource (e.g., a PUT) MAY include an + If-Match header field to signal that the request method MUST NOT be + applied if the entity corresponding to the If-Match value (a single + entity tag) is no longer a representation of that resource. This + allows the user to indicate that they do not wish the request to be + successful if the resource has been changed without their knowledge. + Examples: + + If-Match: "xyzzy" + If-Match: "xyzzy", "r2d2xxxx", "c3piozzzz" + If-Match: * + + The result of a request having both an If-Match header field and + either an If-None-Match or an If-Modified-Since header fields is + undefined by this specification. + +14.25 If-Modified-Since + + The If-Modified-Since request-header field is used with a method to + make it conditional: if the requested variant has not been modified + since the time specified in this field, an entity will not be + returned from the server; instead, a 304 (not modified) response will + be returned without any message-body. + + If-Modified-Since = "If-Modified-Since" ":" HTTP-date + + + +Fielding, et al. Standards Track [Page 130] + +RFC 2616 HTTP/1.1 June 1999 + + + An example of the field is: + + If-Modified-Since: Sat, 29 Oct 1994 19:43:31 GMT + + A GET method with an If-Modified-Since header and no Range header + requests that the identified entity be transferred only if it has + been modified since the date given by the If-Modified-Since header. + The algorithm for determining this includes the following cases: + + a) If the request would normally result in anything other than a + 200 (OK) status, or if the passed If-Modified-Since date is + invalid, the response is exactly the same as for a normal GET. + A date which is later than the server's current time is + invalid. + + b) If the variant has been modified since the If-Modified-Since + date, the response is exactly the same as for a normal GET. + + c) If the variant has not been modified since a valid If- + Modified-Since date, the server SHOULD return a 304 (Not + Modified) response. + + The purpose of this feature is to allow efficient updates of cached + information with a minimum amount of transaction overhead. + + Note: The Range request-header field modifies the meaning of If- + Modified-Since; see section 14.35 for full details. + + Note: If-Modified-Since times are interpreted by the server, whose + clock might not be synchronized with the client. + + Note: When handling an If-Modified-Since header field, some + servers will use an exact date comparison function, rather than a + less-than function, for deciding whether to send a 304 (Not + Modified) response. To get best results when sending an If- + Modified-Since header field for cache validation, clients are + advised to use the exact date string received in a previous Last- + Modified header field whenever possible. + + Note: If a client uses an arbitrary date in the If-Modified-Since + header instead of a date taken from the Last-Modified header for + the same request, the client should be aware of the fact that this + date is interpreted in the server's understanding of time. The + client should consider unsynchronized clocks and rounding problems + due to the different encodings of time between the client and + server. This includes the possibility of race conditions if the + document has changed between the time it was first requested and + the If-Modified-Since date of a subsequent request, and the + + + +Fielding, et al. Standards Track [Page 131] + +RFC 2616 HTTP/1.1 June 1999 + + + possibility of clock-skew-related problems if the If-Modified- + Since date is derived from the client's clock without correction + to the server's clock. Corrections for different time bases + between client and server are at best approximate due to network + latency. + + The result of a request having both an If-Modified-Since header field + and either an If-Match or an If-Unmodified-Since header fields is + undefined by this specification. + +14.26 If-None-Match + + The If-None-Match request-header field is used with a method to make + it conditional. A client that has one or more entities previously + obtained from the resource can verify that none of those entities is + current by including a list of their associated entity tags in the + If-None-Match header field. The purpose of this feature is to allow + efficient updates of cached information with a minimum amount of + transaction overhead. It is also used to prevent a method (e.g. PUT) + from inadvertently modifying an existing resource when the client + believes that the resource does not exist. + + As a special case, the value "*" matches any current entity of the + resource. + + If-None-Match = "If-None-Match" ":" ( "*" | 1#entity-tag ) + + If any of the entity tags match the entity tag of the entity that + would have been returned in the response to a similar GET request + (without the If-None-Match header) on that resource, or if "*" is + given and any current entity exists for that resource, then the + server MUST NOT perform the requested method, unless required to do + so because the resource's modification date fails to match that + supplied in an If-Modified-Since header field in the request. + Instead, if the request method was GET or HEAD, the server SHOULD + respond with a 304 (Not Modified) response, including the cache- + related header fields (particularly ETag) of one of the entities that + matched. For all other request methods, the server MUST respond with + a status of 412 (Precondition Failed). + + See section 13.3.3 for rules on how to determine if two entities tags + match. The weak comparison function can only be used with GET or HEAD + requests. + + + + + + + + +Fielding, et al. Standards Track [Page 132] + +RFC 2616 HTTP/1.1 June 1999 + + + If none of the entity tags match, then the server MAY perform the + requested method as if the If-None-Match header field did not exist, + but MUST also ignore any If-Modified-Since header field(s) in the + request. That is, if no entity tags match, then the server MUST NOT + return a 304 (Not Modified) response. + + If the request would, without the If-None-Match header field, result + in anything other than a 2xx or 304 status, then the If-None-Match + header MUST be ignored. (See section 13.3.4 for a discussion of + server behavior when both If-Modified-Since and If-None-Match appear + in the same request.) + + The meaning of "If-None-Match: *" is that the method MUST NOT be + performed if the representation selected by the origin server (or by + a cache, possibly using the Vary mechanism, see section 14.44) + exists, and SHOULD be performed if the representation does not exist. + This feature is intended to be useful in preventing races between PUT + operations. + + Examples: + + If-None-Match: "xyzzy" + If-None-Match: W/"xyzzy" + If-None-Match: "xyzzy", "r2d2xxxx", "c3piozzzz" + If-None-Match: W/"xyzzy", W/"r2d2xxxx", W/"c3piozzzz" + If-None-Match: * + + The result of a request having both an If-None-Match header field and + either an If-Match or an If-Unmodified-Since header fields is + undefined by this specification. + +14.27 If-Range + + If a client has a partial copy of an entity in its cache, and wishes + to have an up-to-date copy of the entire entity in its cache, it + could use the Range request-header with a conditional GET (using + either or both of If-Unmodified-Since and If-Match.) However, if the + condition fails because the entity has been modified, the client + would then have to make a second request to obtain the entire current + entity-body. + + The If-Range header allows a client to "short-circuit" the second + request. Informally, its meaning is `if the entity is unchanged, send + me the part(s) that I am missing; otherwise, send me the entire new + entity'. + + If-Range = "If-Range" ":" ( entity-tag | HTTP-date ) + + + + +Fielding, et al. Standards Track [Page 133] + +RFC 2616 HTTP/1.1 June 1999 + + + If the client has no entity tag for an entity, but does have a Last- + Modified date, it MAY use that date in an If-Range header. (The + server can distinguish between a valid HTTP-date and any form of + entity-tag by examining no more than two characters.) The If-Range + header SHOULD only be used together with a Range header, and MUST be + ignored if the request does not include a Range header, or if the + server does not support the sub-range operation. + + If the entity tag given in the If-Range header matches the current + entity tag for the entity, then the server SHOULD provide the + specified sub-range of the entity using a 206 (Partial content) + response. If the entity tag does not match, then the server SHOULD + return the entire entity using a 200 (OK) response. + +14.28 If-Unmodified-Since + + The If-Unmodified-Since request-header field is used with a method to + make it conditional. If the requested resource has not been modified + since the time specified in this field, the server SHOULD perform the + requested operation as if the If-Unmodified-Since header were not + present. + + If the requested variant has been modified since the specified time, + the server MUST NOT perform the requested operation, and MUST return + a 412 (Precondition Failed). + + If-Unmodified-Since = "If-Unmodified-Since" ":" HTTP-date + + An example of the field is: + + If-Unmodified-Since: Sat, 29 Oct 1994 19:43:31 GMT + + If the request normally (i.e., without the If-Unmodified-Since + header) would result in anything other than a 2xx or 412 status, the + If-Unmodified-Since header SHOULD be ignored. + + If the specified date is invalid, the header is ignored. + + The result of a request having both an If-Unmodified-Since header + field and either an If-None-Match or an If-Modified-Since header + fields is undefined by this specification. + +14.29 Last-Modified + + The Last-Modified entity-header field indicates the date and time at + which the origin server believes the variant was last modified. + + Last-Modified = "Last-Modified" ":" HTTP-date + + + +Fielding, et al. Standards Track [Page 134] + +RFC 2616 HTTP/1.1 June 1999 + + + An example of its use is + + Last-Modified: Tue, 15 Nov 1994 12:45:26 GMT + + The exact meaning of this header field depends on the implementation + of the origin server and the nature of the original resource. For + files, it may be just the file system last-modified time. For + entities with dynamically included parts, it may be the most recent + of the set of last-modify times for its component parts. For database + gateways, it may be the last-update time stamp of the record. For + virtual objects, it may be the last time the internal state changed. + + An origin server MUST NOT send a Last-Modified date which is later + than the server's time of message origination. In such cases, where + the resource's last modification would indicate some time in the + future, the server MUST replace that date with the message + origination date. + + An origin server SHOULD obtain the Last-Modified value of the entity + as close as possible to the time that it generates the Date value of + its response. This allows a recipient to make an accurate assessment + of the entity's modification time, especially if the entity changes + near the time that the response is generated. + + HTTP/1.1 servers SHOULD send Last-Modified whenever feasible. + +14.30 Location + + The Location response-header field is used to redirect the recipient + to a location other than the Request-URI for completion of the + request or identification of a new resource. For 201 (Created) + responses, the Location is that of the new resource which was created + by the request. For 3xx responses, the location SHOULD indicate the + server's preferred URI for automatic redirection to the resource. The + field value consists of a single absolute URI. + + Location = "Location" ":" absoluteURI + + An example is: + + Location: http://www.w3.org/pub/WWW/People.html + + Note: The Content-Location header field (section 14.14) differs + from Location in that the Content-Location identifies the original + location of the entity enclosed in the request. It is therefore + possible for a response to contain header fields for both Location + and Content-Location. Also see section 13.10 for cache + requirements of some methods. + + + +Fielding, et al. Standards Track [Page 135] + +RFC 2616 HTTP/1.1 June 1999 + + +14.31 Max-Forwards + + The Max-Forwards request-header field provides a mechanism with the + TRACE (section 9.8) and OPTIONS (section 9.2) methods to limit the + number of proxies or gateways that can forward the request to the + next inbound server. This can be useful when the client is attempting + to trace a request chain which appears to be failing or looping in + mid-chain. + + Max-Forwards = "Max-Forwards" ":" 1*DIGIT + + The Max-Forwards value is a decimal integer indicating the remaining + number of times this request message may be forwarded. + + Each proxy or gateway recipient of a TRACE or OPTIONS request + containing a Max-Forwards header field MUST check and update its + value prior to forwarding the request. If the received value is zero + (0), the recipient MUST NOT forward the request; instead, it MUST + respond as the final recipient. If the received Max-Forwards value is + greater than zero, then the forwarded message MUST contain an updated + Max-Forwards field with a value decremented by one (1). + + The Max-Forwards header field MAY be ignored for all other methods + defined by this specification and for any extension methods for which + it is not explicitly referred to as part of that method definition. + +14.32 Pragma + + The Pragma general-header field is used to include implementation- + specific directives that might apply to any recipient along the + request/response chain. All pragma directives specify optional + behavior from the viewpoint of the protocol; however, some systems + MAY require that behavior be consistent with the directives. + + Pragma = "Pragma" ":" 1#pragma-directive + pragma-directive = "no-cache" | extension-pragma + extension-pragma = token [ "=" ( token | quoted-string ) ] + + When the no-cache directive is present in a request message, an + application SHOULD forward the request toward the origin server even + if it has a cached copy of what is being requested. This pragma + directive has the same semantics as the no-cache cache-directive (see + section 14.9) and is defined here for backward compatibility with + HTTP/1.0. Clients SHOULD include both header fields when a no-cache + request is sent to a server not known to be HTTP/1.1 compliant. + + + + + + +Fielding, et al. Standards Track [Page 136] + +RFC 2616 HTTP/1.1 June 1999 + + + Pragma directives MUST be passed through by a proxy or gateway + application, regardless of their significance to that application, + since the directives might be applicable to all recipients along the + request/response chain. It is not possible to specify a pragma for a + specific recipient; however, any pragma directive not relevant to a + recipient SHOULD be ignored by that recipient. + + HTTP/1.1 caches SHOULD treat "Pragma: no-cache" as if the client had + sent "Cache-Control: no-cache". No new Pragma directives will be + defined in HTTP. + + Note: because the meaning of "Pragma: no-cache as a response + header field is not actually specified, it does not provide a + reliable replacement for "Cache-Control: no-cache" in a response + +14.33 Proxy-Authenticate + + The Proxy-Authenticate response-header field MUST be included as part + of a 407 (Proxy Authentication Required) response. The field value + consists of a challenge that indicates the authentication scheme and + parameters applicable to the proxy for this Request-URI. + + Proxy-Authenticate = "Proxy-Authenticate" ":" 1#challenge + + The HTTP access authentication process is described in "HTTP + Authentication: Basic and Digest Access Authentication" [43]. Unlike + WWW-Authenticate, the Proxy-Authenticate header field applies only to + the current connection and SHOULD NOT be passed on to downstream + clients. However, an intermediate proxy might need to obtain its own + credentials by requesting them from the downstream client, which in + some circumstances will appear as if the proxy is forwarding the + Proxy-Authenticate header field. + +14.34 Proxy-Authorization + + The Proxy-Authorization request-header field allows the client to + identify itself (or its user) to a proxy which requires + authentication. The Proxy-Authorization field value consists of + credentials containing the authentication information of the user + agent for the proxy and/or realm of the resource being requested. + + Proxy-Authorization = "Proxy-Authorization" ":" credentials + + The HTTP access authentication process is described in "HTTP + Authentication: Basic and Digest Access Authentication" [43] . Unlike + Authorization, the Proxy-Authorization header field applies only to + the next outbound proxy that demanded authentication using the Proxy- + Authenticate field. When multiple proxies are used in a chain, the + + + +Fielding, et al. Standards Track [Page 137] + +RFC 2616 HTTP/1.1 June 1999 + + + Proxy-Authorization header field is consumed by the first outbound + proxy that was expecting to receive credentials. A proxy MAY relay + the credentials from the client request to the next proxy if that is + the mechanism by which the proxies cooperatively authenticate a given + request. + +14.35 Range + +14.35.1 Byte Ranges + + Since all HTTP entities are represented in HTTP messages as sequences + of bytes, the concept of a byte range is meaningful for any HTTP + entity. (However, not all clients and servers need to support byte- + range operations.) + + Byte range specifications in HTTP apply to the sequence of bytes in + the entity-body (not necessarily the same as the message-body). + + A byte range operation MAY specify a single range of bytes, or a set + of ranges within a single entity. + + ranges-specifier = byte-ranges-specifier + byte-ranges-specifier = bytes-unit "=" byte-range-set + byte-range-set = 1#( byte-range-spec | suffix-byte-range-spec ) + byte-range-spec = first-byte-pos "-" [last-byte-pos] + first-byte-pos = 1*DIGIT + last-byte-pos = 1*DIGIT + + The first-byte-pos value in a byte-range-spec gives the byte-offset + of the first byte in a range. The last-byte-pos value gives the + byte-offset of the last byte in the range; that is, the byte + positions specified are inclusive. Byte offsets start at zero. + + If the last-byte-pos value is present, it MUST be greater than or + equal to the first-byte-pos in that byte-range-spec, or the byte- + range-spec is syntactically invalid. The recipient of a byte-range- + set that includes one or more syntactically invalid byte-range-spec + values MUST ignore the header field that includes that byte-range- + set. + + If the last-byte-pos value is absent, or if the value is greater than + or equal to the current length of the entity-body, last-byte-pos is + taken to be equal to one less than the current length of the entity- + body in bytes. + + By its choice of last-byte-pos, a client can limit the number of + bytes retrieved without knowing the size of the entity. + + + + +Fielding, et al. Standards Track [Page 138] + +RFC 2616 HTTP/1.1 June 1999 + + + suffix-byte-range-spec = "-" suffix-length + suffix-length = 1*DIGIT + + A suffix-byte-range-spec is used to specify the suffix of the + entity-body, of a length given by the suffix-length value. (That is, + this form specifies the last N bytes of an entity-body.) If the + entity is shorter than the specified suffix-length, the entire + entity-body is used. + + If a syntactically valid byte-range-set includes at least one byte- + range-spec whose first-byte-pos is less than the current length of + the entity-body, or at least one suffix-byte-range-spec with a non- + zero suffix-length, then the byte-range-set is satisfiable. + Otherwise, the byte-range-set is unsatisfiable. If the byte-range-set + is unsatisfiable, the server SHOULD return a response with a status + of 416 (Requested range not satisfiable). Otherwise, the server + SHOULD return a response with a status of 206 (Partial Content) + containing the satisfiable ranges of the entity-body. + + Examples of byte-ranges-specifier values (assuming an entity-body of + length 10000): + + - The first 500 bytes (byte offsets 0-499, inclusive): bytes=0- + 499 + + - The second 500 bytes (byte offsets 500-999, inclusive): + bytes=500-999 + + - The final 500 bytes (byte offsets 9500-9999, inclusive): + bytes=-500 + + - Or bytes=9500- + + - The first and last bytes only (bytes 0 and 9999): bytes=0-0,-1 + + - Several legal but not canonical specifications of the second 500 + bytes (byte offsets 500-999, inclusive): + bytes=500-600,601-999 + bytes=500-700,601-999 + +14.35.2 Range Retrieval Requests + + HTTP retrieval requests using conditional or unconditional GET + methods MAY request one or more sub-ranges of the entity, instead of + the entire entity, using the Range request header, which applies to + the entity returned as the result of the request: + + Range = "Range" ":" ranges-specifier + + + +Fielding, et al. Standards Track [Page 139] + +RFC 2616 HTTP/1.1 June 1999 + + + A server MAY ignore the Range header. However, HTTP/1.1 origin + servers and intermediate caches ought to support byte ranges when + possible, since Range supports efficient recovery from partially + failed transfers, and supports efficient partial retrieval of large + entities. + + If the server supports the Range header and the specified range or + ranges are appropriate for the entity: + + - The presence of a Range header in an unconditional GET modifies + what is returned if the GET is otherwise successful. In other + words, the response carries a status code of 206 (Partial + Content) instead of 200 (OK). + + - The presence of a Range header in a conditional GET (a request + using one or both of If-Modified-Since and If-None-Match, or + one or both of If-Unmodified-Since and If-Match) modifies what + is returned if the GET is otherwise successful and the + condition is true. It does not affect the 304 (Not Modified) + response returned if the conditional is false. + + In some cases, it might be more appropriate to use the If-Range + header (see section 14.27) in addition to the Range header. + + If a proxy that supports ranges receives a Range request, forwards + the request to an inbound server, and receives an entire entity in + reply, it SHOULD only return the requested range to its client. It + SHOULD store the entire received response in its cache if that is + consistent with its cache allocation policies. + +14.36 Referer + + The Referer[sic] request-header field allows the client to specify, + for the server's benefit, the address (URI) of the resource from + which the Request-URI was obtained (the "referrer", although the + header field is misspelled.) The Referer request-header allows a + server to generate lists of back-links to resources for interest, + logging, optimized caching, etc. It also allows obsolete or mistyped + links to be traced for maintenance. The Referer field MUST NOT be + sent if the Request-URI was obtained from a source that does not have + its own URI, such as input from the user keyboard. + + Referer = "Referer" ":" ( absoluteURI | relativeURI ) + + Example: + + Referer: http://www.w3.org/hypertext/DataSources/Overview.html + + + + +Fielding, et al. Standards Track [Page 140] + +RFC 2616 HTTP/1.1 June 1999 + + + If the field value is a relative URI, it SHOULD be interpreted + relative to the Request-URI. The URI MUST NOT include a fragment. See + section 15.1.3 for security considerations. + +14.37 Retry-After + + The Retry-After response-header field can be used with a 503 (Service + Unavailable) response to indicate how long the service is expected to + be unavailable to the requesting client. This field MAY also be used + with any 3xx (Redirection) response to indicate the minimum time the + user-agent is asked wait before issuing the redirected request. The + value of this field can be either an HTTP-date or an integer number + of seconds (in decimal) after the time of the response. + + Retry-After = "Retry-After" ":" ( HTTP-date | delta-seconds ) + + Two examples of its use are + + Retry-After: Fri, 31 Dec 1999 23:59:59 GMT + Retry-After: 120 + + In the latter example, the delay is 2 minutes. + +14.38 Server + + The Server response-header field contains information about the + software used by the origin server to handle the request. The field + can contain multiple product tokens (section 3.8) and comments + identifying the server and any significant subproducts. The product + tokens are listed in order of their significance for identifying the + application. + + Server = "Server" ":" 1*( product | comment ) + + Example: + + Server: CERN/3.0 libwww/2.17 + + If the response is being forwarded through a proxy, the proxy + application MUST NOT modify the Server response-header. Instead, it + SHOULD include a Via field (as described in section 14.45). + + Note: Revealing the specific software version of the server might + allow the server machine to become more vulnerable to attacks + against software that is known to contain security holes. Server + implementors are encouraged to make this field a configurable + option. + + + + +Fielding, et al. Standards Track [Page 141] + +RFC 2616 HTTP/1.1 June 1999 + + +14.39 TE + + The TE request-header field indicates what extension transfer-codings + it is willing to accept in the response and whether or not it is + willing to accept trailer fields in a chunked transfer-coding. Its + value may consist of the keyword "trailers" and/or a comma-separated + list of extension transfer-coding names with optional accept + parameters (as described in section 3.6). + + TE = "TE" ":" #( t-codings ) + t-codings = "trailers" | ( transfer-extension [ accept-params ] ) + + The presence of the keyword "trailers" indicates that the client is + willing to accept trailer fields in a chunked transfer-coding, as + defined in section 3.6.1. This keyword is reserved for use with + transfer-coding values even though it does not itself represent a + transfer-coding. + + Examples of its use are: + + TE: deflate + TE: + TE: trailers, deflate;q=0.5 + + The TE header field only applies to the immediate connection. + Therefore, the keyword MUST be supplied within a Connection header + field (section 14.10) whenever TE is present in an HTTP/1.1 message. + + A server tests whether a transfer-coding is acceptable, according to + a TE field, using these rules: + + 1. The "chunked" transfer-coding is always acceptable. If the + keyword "trailers" is listed, the client indicates that it is + willing to accept trailer fields in the chunked response on + behalf of itself and any downstream clients. The implication is + that, if given, the client is stating that either all + downstream clients are willing to accept trailer fields in the + forwarded response, or that it will attempt to buffer the + response on behalf of downstream recipients. + + Note: HTTP/1.1 does not define any means to limit the size of a + chunked response such that a client can be assured of buffering + the entire response. + + 2. If the transfer-coding being tested is one of the transfer- + codings listed in the TE field, then it is acceptable unless it + is accompanied by a qvalue of 0. (As defined in section 3.9, a + qvalue of 0 means "not acceptable.") + + + +Fielding, et al. Standards Track [Page 142] + +RFC 2616 HTTP/1.1 June 1999 + + + 3. If multiple transfer-codings are acceptable, then the + acceptable transfer-coding with the highest non-zero qvalue is + preferred. The "chunked" transfer-coding always has a qvalue + of 1. + + If the TE field-value is empty or if no TE field is present, the only + transfer-coding is "chunked". A message with no transfer-coding is + always acceptable. + +14.40 Trailer + + The Trailer general field value indicates that the given set of + header fields is present in the trailer of a message encoded with + chunked transfer-coding. + + Trailer = "Trailer" ":" 1#field-name + + An HTTP/1.1 message SHOULD include a Trailer header field in a + message using chunked transfer-coding with a non-empty trailer. Doing + so allows the recipient to know which header fields to expect in the + trailer. + + If no Trailer header field is present, the trailer SHOULD NOT include + any header fields. See section 3.6.1 for restrictions on the use of + trailer fields in a "chunked" transfer-coding. + + Message header fields listed in the Trailer header field MUST NOT + include the following header fields: + + . Transfer-Encoding + + . Content-Length + + . Trailer + +14.41 Transfer-Encoding + + The Transfer-Encoding general-header field indicates what (if any) + type of transformation has been applied to the message body in order + to safely transfer it between the sender and the recipient. This + differs from the content-coding in that the transfer-coding is a + property of the message, not of the entity. + + Transfer-Encoding = "Transfer-Encoding" ":" 1#transfer-coding + + Transfer-codings are defined in section 3.6. An example is: + + Transfer-Encoding: chunked + + + +Fielding, et al. Standards Track [Page 143] + +RFC 2616 HTTP/1.1 June 1999 + + + If multiple encodings have been applied to an entity, the transfer- + codings MUST be listed in the order in which they were applied. + Additional information about the encoding parameters MAY be provided + by other entity-header fields not defined by this specification. + + Many older HTTP/1.0 applications do not understand the Transfer- + Encoding header. + +14.42 Upgrade + + The Upgrade general-header allows the client to specify what + additional communication protocols it supports and would like to use + if the server finds it appropriate to switch protocols. The server + MUST use the Upgrade header field within a 101 (Switching Protocols) + response to indicate which protocol(s) are being switched. + + Upgrade = "Upgrade" ":" 1#product + + For example, + + Upgrade: HTTP/2.0, SHTTP/1.3, IRC/6.9, RTA/x11 + + The Upgrade header field is intended to provide a simple mechanism + for transition from HTTP/1.1 to some other, incompatible protocol. It + does so by allowing the client to advertise its desire to use another + protocol, such as a later version of HTTP with a higher major version + number, even though the current request has been made using HTTP/1.1. + This eases the difficult transition between incompatible protocols by + allowing the client to initiate a request in the more commonly + supported protocol while indicating to the server that it would like + to use a "better" protocol if available (where "better" is determined + by the server, possibly according to the nature of the method and/or + resource being requested). + + The Upgrade header field only applies to switching application-layer + protocols upon the existing transport-layer connection. Upgrade + cannot be used to insist on a protocol change; its acceptance and use + by the server is optional. The capabilities and nature of the + application-layer communication after the protocol change is entirely + dependent upon the new protocol chosen, although the first action + after changing the protocol MUST be a response to the initial HTTP + request containing the Upgrade header field. + + The Upgrade header field only applies to the immediate connection. + Therefore, the upgrade keyword MUST be supplied within a Connection + header field (section 14.10) whenever Upgrade is present in an + HTTP/1.1 message. + + + + +Fielding, et al. Standards Track [Page 144] + +RFC 2616 HTTP/1.1 June 1999 + + + The Upgrade header field cannot be used to indicate a switch to a + protocol on a different connection. For that purpose, it is more + appropriate to use a 301, 302, 303, or 305 redirection response. + + This specification only defines the protocol name "HTTP" for use by + the family of Hypertext Transfer Protocols, as defined by the HTTP + version rules of section 3.1 and future updates to this + specification. Any token can be used as a protocol name; however, it + will only be useful if both the client and server associate the name + with the same protocol. + +14.43 User-Agent + + The User-Agent request-header field contains information about the + user agent originating the request. This is for statistical purposes, + the tracing of protocol violations, and automated recognition of user + agents for the sake of tailoring responses to avoid particular user + agent limitations. User agents SHOULD include this field with + requests. The field can contain multiple product tokens (section 3.8) + and comments identifying the agent and any subproducts which form a + significant part of the user agent. By convention, the product tokens + are listed in order of their significance for identifying the + application. + + User-Agent = "User-Agent" ":" 1*( product | comment ) + + Example: + + User-Agent: CERN-LineMode/2.15 libwww/2.17b3 + +14.44 Vary + + The Vary field value indicates the set of request-header fields that + fully determines, while the response is fresh, whether a cache is + permitted to use the response to reply to a subsequent request + without revalidation. For uncacheable or stale responses, the Vary + field value advises the user agent about the criteria that were used + to select the representation. A Vary field value of "*" implies that + a cache cannot determine from the request headers of a subsequent + request whether this response is the appropriate representation. See + section 13.6 for use of the Vary header field by caches. + + Vary = "Vary" ":" ( "*" | 1#field-name ) + + An HTTP/1.1 server SHOULD include a Vary header field with any + cacheable response that is subject to server-driven negotiation. + Doing so allows a cache to properly interpret future requests on that + resource and informs the user agent about the presence of negotiation + + + +Fielding, et al. Standards Track [Page 145] + +RFC 2616 HTTP/1.1 June 1999 + + + on that resource. A server MAY include a Vary header field with a + non-cacheable response that is subject to server-driven negotiation, + since this might provide the user agent with useful information about + the dimensions over which the response varies at the time of the + response. + + A Vary field value consisting of a list of field-names signals that + the representation selected for the response is based on a selection + algorithm which considers ONLY the listed request-header field values + in selecting the most appropriate representation. A cache MAY assume + that the same selection will be made for future requests with the + same values for the listed field names, for the duration of time for + which the response is fresh. + + The field-names given are not limited to the set of standard + request-header fields defined by this specification. Field names are + case-insensitive. + + A Vary field value of "*" signals that unspecified parameters not + limited to the request-headers (e.g., the network address of the + client), play a role in the selection of the response representation. + The "*" value MUST NOT be generated by a proxy server; it may only be + generated by an origin server. + +14.45 Via + + The Via general-header field MUST be used by gateways and proxies to + indicate the intermediate protocols and recipients between the user + agent and the server on requests, and between the origin server and + the client on responses. It is analogous to the "Received" field of + RFC 822 [9] and is intended to be used for tracking message forwards, + avoiding request loops, and identifying the protocol capabilities of + all senders along the request/response chain. + + Via = "Via" ":" 1#( received-protocol received-by [ comment ] ) + received-protocol = [ protocol-name "/" ] protocol-version + protocol-name = token + protocol-version = token + received-by = ( host [ ":" port ] ) | pseudonym + pseudonym = token + + The received-protocol indicates the protocol version of the message + received by the server or client along each segment of the + request/response chain. The received-protocol version is appended to + the Via field value when the message is forwarded so that information + about the protocol capabilities of upstream applications remains + visible to all recipients. + + + + +Fielding, et al. Standards Track [Page 146] + +RFC 2616 HTTP/1.1 June 1999 + + + The protocol-name is optional if and only if it would be "HTTP". The + received-by field is normally the host and optional port number of a + recipient server or client that subsequently forwarded the message. + However, if the real host is considered to be sensitive information, + it MAY be replaced by a pseudonym. If the port is not given, it MAY + be assumed to be the default port of the received-protocol. + + Multiple Via field values represents each proxy or gateway that has + forwarded the message. Each recipient MUST append its information + such that the end result is ordered according to the sequence of + forwarding applications. + + Comments MAY be used in the Via header field to identify the software + of the recipient proxy or gateway, analogous to the User-Agent and + Server header fields. However, all comments in the Via field are + optional and MAY be removed by any recipient prior to forwarding the + message. + + For example, a request message could be sent from an HTTP/1.0 user + agent to an internal proxy code-named "fred", which uses HTTP/1.1 to + forward the request to a public proxy at nowhere.com, which completes + the request by forwarding it to the origin server at www.ics.uci.edu. + The request received by www.ics.uci.edu would then have the following + Via header field: + + Via: 1.0 fred, 1.1 nowhere.com (Apache/1.1) + + Proxies and gateways used as a portal through a network firewall + SHOULD NOT, by default, forward the names and ports of hosts within + the firewall region. This information SHOULD only be propagated if + explicitly enabled. If not enabled, the received-by host of any host + behind the firewall SHOULD be replaced by an appropriate pseudonym + for that host. + + For organizations that have strong privacy requirements for hiding + internal structures, a proxy MAY combine an ordered subsequence of + Via header field entries with identical received-protocol values into + a single such entry. For example, + + Via: 1.0 ricky, 1.1 ethel, 1.1 fred, 1.0 lucy + + could be collapsed to + + Via: 1.0 ricky, 1.1 mertz, 1.0 lucy + + + + + + + +Fielding, et al. Standards Track [Page 147] + +RFC 2616 HTTP/1.1 June 1999 + + + Applications SHOULD NOT combine multiple entries unless they are all + under the same organizational control and the hosts have already been + replaced by pseudonyms. Applications MUST NOT combine entries which + have different received-protocol values. + +14.46 Warning + + The Warning general-header field is used to carry additional + information about the status or transformation of a message which + might not be reflected in the message. This information is typically + used to warn about a possible lack of semantic transparency from + caching operations or transformations applied to the entity body of + the message. + + Warning headers are sent with responses using: + + Warning = "Warning" ":" 1#warning-value + + warning-value = warn-code SP warn-agent SP warn-text + [SP warn-date] + + warn-code = 3DIGIT + warn-agent = ( host [ ":" port ] ) | pseudonym + ; the name or pseudonym of the server adding + ; the Warning header, for use in debugging + warn-text = quoted-string + warn-date = <"> HTTP-date <"> + + A response MAY carry more than one Warning header. + + The warn-text SHOULD be in a natural language and character set that + is most likely to be intelligible to the human user receiving the + response. This decision MAY be based on any available knowledge, such + as the location of the cache or user, the Accept-Language field in a + request, the Content-Language field in a response, etc. The default + language is English and the default character set is ISO-8859-1. + + If a character set other than ISO-8859-1 is used, it MUST be encoded + in the warn-text using the method described in RFC 2047 [14]. + + Warning headers can in general be applied to any message, however + some specific warn-codes are specific to caches and can only be + applied to response messages. New Warning headers SHOULD be added + after any existing Warning headers. A cache MUST NOT delete any + Warning header that it received with a message. However, if a cache + successfully validates a cache entry, it SHOULD remove any Warning + headers previously attached to that entry except as specified for + + + + +Fielding, et al. Standards Track [Page 148] + +RFC 2616 HTTP/1.1 June 1999 + + + specific Warning codes. It MUST then add any Warning headers received + in the validating response. In other words, Warning headers are those + that would be attached to the most recent relevant response. + + When multiple Warning headers are attached to a response, the user + agent ought to inform the user of as many of them as possible, in the + order that they appear in the response. If it is not possible to + inform the user of all of the warnings, the user agent SHOULD follow + these heuristics: + + - Warnings that appear early in the response take priority over + those appearing later in the response. + + - Warnings in the user's preferred character set take priority + over warnings in other character sets but with identical warn- + codes and warn-agents. + + Systems that generate multiple Warning headers SHOULD order them with + this user agent behavior in mind. + + Requirements for the behavior of caches with respect to Warnings are + stated in section 13.1.2. + + This is a list of the currently-defined warn-codes, each with a + recommended warn-text in English, and a description of its meaning. + + 110 Response is stale + MUST be included whenever the returned response is stale. + + 111 Revalidation failed + MUST be included if a cache returns a stale response because an + attempt to revalidate the response failed, due to an inability to + reach the server. + + 112 Disconnected operation + SHOULD be included if the cache is intentionally disconnected from + the rest of the network for a period of time. + + 113 Heuristic expiration + MUST be included if the cache heuristically chose a freshness + lifetime greater than 24 hours and the response's age is greater + than 24 hours. + + 199 Miscellaneous warning + The warning text MAY include arbitrary information to be presented + to a human user, or logged. A system receiving this warning MUST + NOT take any automated action, besides presenting the warning to + the user. + + + +Fielding, et al. Standards Track [Page 149] + +RFC 2616 HTTP/1.1 June 1999 + + + 214 Transformation applied + MUST be added by an intermediate cache or proxy if it applies any + transformation changing the content-coding (as specified in the + Content-Encoding header) or media-type (as specified in the + Content-Type header) of the response, or the entity-body of the + response, unless this Warning code already appears in the response. + + 299 Miscellaneous persistent warning + The warning text MAY include arbitrary information to be presented + to a human user, or logged. A system receiving this warning MUST + NOT take any automated action. + + If an implementation sends a message with one or more Warning headers + whose version is HTTP/1.0 or lower, then the sender MUST include in + each warning-value a warn-date that matches the date in the response. + + If an implementation receives a message with a warning-value that + includes a warn-date, and that warn-date is different from the Date + value in the response, then that warning-value MUST be deleted from + the message before storing, forwarding, or using it. (This prevents + bad consequences of naive caching of Warning header fields.) If all + of the warning-values are deleted for this reason, the Warning header + MUST be deleted as well. + +14.47 WWW-Authenticate + + The WWW-Authenticate response-header field MUST be included in 401 + (Unauthorized) response messages. The field value consists of at + least one challenge that indicates the authentication scheme(s) and + parameters applicable to the Request-URI. + + WWW-Authenticate = "WWW-Authenticate" ":" 1#challenge + + The HTTP access authentication process is described in "HTTP + Authentication: Basic and Digest Access Authentication" [43]. User + agents are advised to take special care in parsing the WWW- + Authenticate field value as it might contain more than one challenge, + or if more than one WWW-Authenticate header field is provided, the + contents of a challenge itself can contain a comma-separated list of + authentication parameters. + +15 Security Considerations + + This section is meant to inform application developers, information + providers, and users of the security limitations in HTTP/1.1 as + described by this document. The discussion does not include + definitive solutions to the problems revealed, though it does make + some suggestions for reducing security risks. + + + +Fielding, et al. Standards Track [Page 150] + +RFC 2616 HTTP/1.1 June 1999 + + +15.1 Personal Information + + HTTP clients are often privy to large amounts of personal information + (e.g. the user's name, location, mail address, passwords, encryption + keys, etc.), and SHOULD be very careful to prevent unintentional + leakage of this information via the HTTP protocol to other sources. + We very strongly recommend that a convenient interface be provided + for the user to control dissemination of such information, and that + designers and implementors be particularly careful in this area. + History shows that errors in this area often create serious security + and/or privacy problems and generate highly adverse publicity for the + implementor's company. + +15.1.1 Abuse of Server Log Information + + A server is in the position to save personal data about a user's + requests which might identify their reading patterns or subjects of + interest. This information is clearly confidential in nature and its + handling can be constrained by law in certain countries. People using + the HTTP protocol to provide data are responsible for ensuring that + such material is not distributed without the permission of any + individuals that are identifiable by the published results. + +15.1.2 Transfer of Sensitive Information + + Like any generic data transfer protocol, HTTP cannot regulate the + content of the data that is transferred, nor is there any a priori + method of determining the sensitivity of any particular piece of + information within the context of any given request. Therefore, + applications SHOULD supply as much control over this information as + possible to the provider of that information. Four header fields are + worth special mention in this context: Server, Via, Referer and From. + + Revealing the specific software version of the server might allow the + server machine to become more vulnerable to attacks against software + that is known to contain security holes. Implementors SHOULD make the + Server header field a configurable option. + + Proxies which serve as a portal through a network firewall SHOULD + take special precautions regarding the transfer of header information + that identifies the hosts behind the firewall. In particular, they + SHOULD remove, or replace with sanitized versions, any Via fields + generated behind the firewall. + + The Referer header allows reading patterns to be studied and reverse + links drawn. Although it can be very useful, its power can be abused + if user details are not separated from the information contained in + + + + +Fielding, et al. Standards Track [Page 151] + +RFC 2616 HTTP/1.1 June 1999 + + + the Referer. Even when the personal information has been removed, the + Referer header might indicate a private document's URI whose + publication would be inappropriate. + + The information sent in the From field might conflict with the user's + privacy interests or their site's security policy, and hence it + SHOULD NOT be transmitted without the user being able to disable, + enable, and modify the contents of the field. The user MUST be able + to set the contents of this field within a user preference or + application defaults configuration. + + We suggest, though do not require, that a convenient toggle interface + be provided for the user to enable or disable the sending of From and + Referer information. + + The User-Agent (section 14.43) or Server (section 14.38) header + fields can sometimes be used to determine that a specific client or + server have a particular security hole which might be exploited. + Unfortunately, this same information is often used for other valuable + purposes for which HTTP currently has no better mechanism. + +15.1.3 Encoding Sensitive Information in URI's + + Because the source of a link might be private information or might + reveal an otherwise private information source, it is strongly + recommended that the user be able to select whether or not the + Referer field is sent. For example, a browser client could have a + toggle switch for browsing openly/anonymously, which would + respectively enable/disable the sending of Referer and From + information. + + Clients SHOULD NOT include a Referer header field in a (non-secure) + HTTP request if the referring page was transferred with a secure + protocol. + + Authors of services which use the HTTP protocol SHOULD NOT use GET + based forms for the submission of sensitive data, because this will + cause this data to be encoded in the Request-URI. Many existing + servers, proxies, and user agents will log the request URI in some + place where it might be visible to third parties. Servers can use + POST-based form submission instead + +15.1.4 Privacy Issues Connected to Accept Headers + + Accept request-headers can reveal information about the user to all + servers which are accessed. The Accept-Language header in particular + can reveal information the user would consider to be of a private + nature, because the understanding of particular languages is often + + + +Fielding, et al. Standards Track [Page 152] + +RFC 2616 HTTP/1.1 June 1999 + + + strongly correlated to the membership of a particular ethnic group. + User agents which offer the option to configure the contents of an + Accept-Language header to be sent in every request are strongly + encouraged to let the configuration process include a message which + makes the user aware of the loss of privacy involved. + + An approach that limits the loss of privacy would be for a user agent + to omit the sending of Accept-Language headers by default, and to ask + the user whether or not to start sending Accept-Language headers to a + server if it detects, by looking for any Vary response-header fields + generated by the server, that such sending could improve the quality + of service. + + Elaborate user-customized accept header fields sent in every request, + in particular if these include quality values, can be used by servers + as relatively reliable and long-lived user identifiers. Such user + identifiers would allow content providers to do click-trail tracking, + and would allow collaborating content providers to match cross-server + click-trails or form submissions of individual users. Note that for + many users not behind a proxy, the network address of the host + running the user agent will also serve as a long-lived user + identifier. In environments where proxies are used to enhance + privacy, user agents ought to be conservative in offering accept + header configuration options to end users. As an extreme privacy + measure, proxies could filter the accept headers in relayed requests. + General purpose user agents which provide a high degree of header + configurability SHOULD warn users about the loss of privacy which can + be involved. + +15.2 Attacks Based On File and Path Names + + Implementations of HTTP origin servers SHOULD be careful to restrict + the documents returned by HTTP requests to be only those that were + intended by the server administrators. If an HTTP server translates + HTTP URIs directly into file system calls, the server MUST take + special care not to serve files that were not intended to be + delivered to HTTP clients. For example, UNIX, Microsoft Windows, and + other operating systems use ".." as a path component to indicate a + directory level above the current one. On such a system, an HTTP + server MUST disallow any such construct in the Request-URI if it + would otherwise allow access to a resource outside those intended to + be accessible via the HTTP server. Similarly, files intended for + reference only internally to the server (such as access control + files, configuration files, and script code) MUST be protected from + inappropriate retrieval, since they might contain sensitive + information. Experience has shown that minor bugs in such HTTP server + implementations have turned into security risks. + + + + +Fielding, et al. Standards Track [Page 153] + +RFC 2616 HTTP/1.1 June 1999 + + +15.3 DNS Spoofing + + Clients using HTTP rely heavily on the Domain Name Service, and are + thus generally prone to security attacks based on the deliberate + mis-association of IP addresses and DNS names. Clients need to be + cautious in assuming the continuing validity of an IP number/DNS name + association. + + In particular, HTTP clients SHOULD rely on their name resolver for + confirmation of an IP number/DNS name association, rather than + caching the result of previous host name lookups. Many platforms + already can cache host name lookups locally when appropriate, and + they SHOULD be configured to do so. It is proper for these lookups to + be cached, however, only when the TTL (Time To Live) information + reported by the name server makes it likely that the cached + information will remain useful. + + If HTTP clients cache the results of host name lookups in order to + achieve a performance improvement, they MUST observe the TTL + information reported by DNS. + + If HTTP clients do not observe this rule, they could be spoofed when + a previously-accessed server's IP address changes. As network + renumbering is expected to become increasingly common [24], the + possibility of this form of attack will grow. Observing this + requirement thus reduces this potential security vulnerability. + + This requirement also improves the load-balancing behavior of clients + for replicated servers using the same DNS name and reduces the + likelihood of a user's experiencing failure in accessing sites which + use that strategy. + +15.4 Location Headers and Spoofing + + If a single server supports multiple organizations that do not trust + one another, then it MUST check the values of Location and Content- + Location headers in responses that are generated under control of + said organizations to make sure that they do not attempt to + invalidate resources over which they have no authority. + +15.5 Content-Disposition Issues + + RFC 1806 [35], from which the often implemented Content-Disposition + (see section 19.5.1) header in HTTP is derived, has a number of very + serious security considerations. Content-Disposition is not part of + the HTTP standard, but since it is widely implemented, we are + documenting its use and risks for implementors. See RFC 2183 [49] + (which updates RFC 1806) for details. + + + +Fielding, et al. Standards Track [Page 154] + +RFC 2616 HTTP/1.1 June 1999 + + +15.6 Authentication Credentials and Idle Clients + + Existing HTTP clients and user agents typically retain authentication + information indefinitely. HTTP/1.1. does not provide a method for a + server to direct clients to discard these cached credentials. This is + a significant defect that requires further extensions to HTTP. + Circumstances under which credential caching can interfere with the + application's security model include but are not limited to: + + - Clients which have been idle for an extended period following + which the server might wish to cause the client to reprompt the + user for credentials. + + - Applications which include a session termination indication + (such as a `logout' or `commit' button on a page) after which + the server side of the application `knows' that there is no + further reason for the client to retain the credentials. + + This is currently under separate study. There are a number of work- + arounds to parts of this problem, and we encourage the use of + password protection in screen savers, idle time-outs, and other + methods which mitigate the security problems inherent in this + problem. In particular, user agents which cache credentials are + encouraged to provide a readily accessible mechanism for discarding + cached credentials under user control. + +15.7 Proxies and Caching + + By their very nature, HTTP proxies are men-in-the-middle, and + represent an opportunity for man-in-the-middle attacks. Compromise of + the systems on which the proxies run can result in serious security + and privacy problems. Proxies have access to security-related + information, personal information about individual users and + organizations, and proprietary information belonging to users and + content providers. A compromised proxy, or a proxy implemented or + configured without regard to security and privacy considerations, + might be used in the commission of a wide range of potential attacks. + + Proxy operators should protect the systems on which proxies run as + they would protect any system that contains or transports sensitive + information. In particular, log information gathered at proxies often + contains highly sensitive personal information, and/or information + about organizations. Log information should be carefully guarded, and + appropriate guidelines for use developed and followed. (Section + 15.1.1). + + + + + + +Fielding, et al. Standards Track [Page 155] + +RFC 2616 HTTP/1.1 June 1999 + + + Caching proxies provide additional potential vulnerabilities, since + the contents of the cache represent an attractive target for + malicious exploitation. Because cache contents persist after an HTTP + request is complete, an attack on the cache can reveal information + long after a user believes that the information has been removed from + the network. Therefore, cache contents should be protected as + sensitive information. + + Proxy implementors should consider the privacy and security + implications of their design and coding decisions, and of the + configuration options they provide to proxy operators (especially the + default configuration). + + Users of a proxy need to be aware that they are no trustworthier than + the people who run the proxy; HTTP itself cannot solve this problem. + + The judicious use of cryptography, when appropriate, may suffice to + protect against a broad range of security and privacy attacks. Such + cryptography is beyond the scope of the HTTP/1.1 specification. + +15.7.1 Denial of Service Attacks on Proxies + + They exist. They are hard to defend against. Research continues. + Beware. + +16 Acknowledgments + + This specification makes heavy use of the augmented BNF and generic + constructs defined by David H. Crocker for RFC 822 [9]. Similarly, it + reuses many of the definitions provided by Nathaniel Borenstein and + Ned Freed for MIME [7]. We hope that their inclusion in this + specification will help reduce past confusion over the relationship + between HTTP and Internet mail message formats. + + The HTTP protocol has evolved considerably over the years. It has + benefited from a large and active developer community--the many + people who have participated on the www-talk mailing list--and it is + that community which has been most responsible for the success of + HTTP and of the World-Wide Web in general. Marc Andreessen, Robert + Cailliau, Daniel W. Connolly, Bob Denny, John Franks, Jean-Francois + Groff, Phillip M. Hallam-Baker, Hakon W. Lie, Ari Luotonen, Rob + McCool, Lou Montulli, Dave Raggett, Tony Sanders, and Marc + VanHeyningen deserve special recognition for their efforts in + defining early aspects of the protocol. + + This document has benefited greatly from the comments of all those + participating in the HTTP-WG. In addition to those already mentioned, + the following individuals have contributed to this specification: + + + +Fielding, et al. Standards Track [Page 156] + +RFC 2616 HTTP/1.1 June 1999 + + + Gary Adams Ross Patterson + Harald Tveit Alvestrand Albert Lunde + Keith Ball John C. Mallery + Brian Behlendorf Jean-Philippe Martin-Flatin + Paul Burchard Mitra + Maurizio Codogno David Morris + Mike Cowlishaw Gavin Nicol + Roman Czyborra Bill Perry + Michael A. Dolan Jeffrey Perry + David J. Fiander Scott Powers + Alan Freier Owen Rees + Marc Hedlund Luigi Rizzo + Greg Herlihy David Robinson + Koen Holtman Marc Salomon + Alex Hopmann Rich Salz + Bob Jernigan Allan M. Schiffman + Shel Kaphan Jim Seidman + Rohit Khare Chuck Shotton + John Klensin Eric W. Sink + Martijn Koster Simon E. Spero + Alexei Kosut Richard N. Taylor + David M. Kristol Robert S. Thau + Daniel LaLiberte Bill (BearHeart) Weinman + Ben Laurie Francois Yergeau + Paul J. Leach Mary Ellen Zurko + Daniel DuBois Josh Cohen + + + Much of the content and presentation of the caching design is due to + suggestions and comments from individuals including: Shel Kaphan, + Paul Leach, Koen Holtman, David Morris, and Larry Masinter. + + Most of the specification of ranges is based on work originally done + by Ari Luotonen and John Franks, with additional input from Steve + Zilles. + + Thanks to the "cave men" of Palo Alto. You know who you are. + + Jim Gettys (the current editor of this document) wishes particularly + to thank Roy Fielding, the previous editor of this document, along + with John Klensin, Jeff Mogul, Paul Leach, Dave Kristol, Koen + Holtman, John Franks, Josh Cohen, Alex Hopmann, Scott Lawrence, and + Larry Masinter for their help. And thanks go particularly to Jeff + Mogul and Scott Lawrence for performing the "MUST/MAY/SHOULD" audit. + + + + + + + +Fielding, et al. Standards Track [Page 157] + +RFC 2616 HTTP/1.1 June 1999 + + + The Apache Group, Anselm Baird-Smith, author of Jigsaw, and Henrik + Frystyk implemented RFC 2068 early, and we wish to thank them for the + discovery of many of the problems that this document attempts to + rectify. + +17 References + + [1] Alvestrand, H., "Tags for the Identification of Languages", RFC + 1766, March 1995. + + [2] Anklesaria, F., McCahill, M., Lindner, P., Johnson, D., Torrey, + D. and B. Alberti, "The Internet Gopher Protocol (a distributed + document search and retrieval protocol)", RFC 1436, March 1993. + + [3] Berners-Lee, T., "Universal Resource Identifiers in WWW", RFC + 1630, June 1994. + + [4] Berners-Lee, T., Masinter, L. and M. McCahill, "Uniform Resource + Locators (URL)", RFC 1738, December 1994. + + [5] Berners-Lee, T. and D. Connolly, "Hypertext Markup Language - + 2.0", RFC 1866, November 1995. + + [6] Berners-Lee, T., Fielding, R. and H. Frystyk, "Hypertext Transfer + Protocol -- HTTP/1.0", RFC 1945, May 1996. + + [7] Freed, N. and N. Borenstein, "Multipurpose Internet Mail + Extensions (MIME) Part One: Format of Internet Message Bodies", + RFC 2045, November 1996. + + [8] Braden, R., "Requirements for Internet Hosts -- Communication + Layers", STD 3, RFC 1123, October 1989. + + [9] Crocker, D., "Standard for The Format of ARPA Internet Text + Messages", STD 11, RFC 822, August 1982. + + [10] Davis, F., Kahle, B., Morris, H., Salem, J., Shen, T., Wang, R., + Sui, J., and M. Grinbaum, "WAIS Interface Protocol Prototype + Functional Specification," (v1.5), Thinking Machines + Corporation, April 1990. + + [11] Fielding, R., "Relative Uniform Resource Locators", RFC 1808, + June 1995. + + [12] Horton, M. and R. Adams, "Standard for Interchange of USENET + Messages", RFC 1036, December 1987. + + + + + +Fielding, et al. Standards Track [Page 158] + +RFC 2616 HTTP/1.1 June 1999 + + + [13] Kantor, B. and P. Lapsley, "Network News Transfer Protocol", RFC + 977, February 1986. + + [14] Moore, K., "MIME (Multipurpose Internet Mail Extensions) Part + Three: Message Header Extensions for Non-ASCII Text", RFC 2047, + November 1996. + + [15] Nebel, E. and L. Masinter, "Form-based File Upload in HTML", RFC + 1867, November 1995. + + [16] Postel, J., "Simple Mail Transfer Protocol", STD 10, RFC 821, + August 1982. + + [17] Postel, J., "Media Type Registration Procedure", RFC 1590, + November 1996. + + [18] Postel, J. and J. Reynolds, "File Transfer Protocol", STD 9, RFC + 959, October 1985. + + [19] Reynolds, J. and J. Postel, "Assigned Numbers", STD 2, RFC 1700, + October 1994. + + [20] Sollins, K. and L. Masinter, "Functional Requirements for + Uniform Resource Names", RFC 1737, December 1994. + + [21] US-ASCII. Coded Character Set - 7-Bit American Standard Code for + Information Interchange. Standard ANSI X3.4-1986, ANSI, 1986. + + [22] ISO-8859. International Standard -- Information Processing -- + 8-bit Single-Byte Coded Graphic Character Sets -- + Part 1: Latin alphabet No. 1, ISO-8859-1:1987. + Part 2: Latin alphabet No. 2, ISO-8859-2, 1987. + Part 3: Latin alphabet No. 3, ISO-8859-3, 1988. + Part 4: Latin alphabet No. 4, ISO-8859-4, 1988. + Part 5: Latin/Cyrillic alphabet, ISO-8859-5, 1988. + Part 6: Latin/Arabic alphabet, ISO-8859-6, 1987. + Part 7: Latin/Greek alphabet, ISO-8859-7, 1987. + Part 8: Latin/Hebrew alphabet, ISO-8859-8, 1988. + Part 9: Latin alphabet No. 5, ISO-8859-9, 1990. + + [23] Meyers, J. and M. Rose, "The Content-MD5 Header Field", RFC + 1864, October 1995. + + [24] Carpenter, B. and Y. Rekhter, "Renumbering Needs Work", RFC + 1900, February 1996. + + [25] Deutsch, P., "GZIP file format specification version 4.3", RFC + 1952, May 1996. + + + +Fielding, et al. Standards Track [Page 159] + +RFC 2616 HTTP/1.1 June 1999 + + + [26] Venkata N. Padmanabhan, and Jeffrey C. Mogul. "Improving HTTP + Latency", Computer Networks and ISDN Systems, v. 28, pp. 25-35, + Dec. 1995. Slightly revised version of paper in Proc. 2nd + International WWW Conference '94: Mosaic and the Web, Oct. 1994, + which is available at + http://www.ncsa.uiuc.edu/SDG/IT94/Proceedings/DDay/mogul/HTTPLat + ency.html. + + [27] Joe Touch, John Heidemann, and Katia Obraczka. "Analysis of HTTP + Performance", , + ISI Research Report ISI/RR-98-463, (original report dated Aug. + 1996), USC/Information Sciences Institute, August 1998. + + [28] Mills, D., "Network Time Protocol (Version 3) Specification, + Implementation and Analysis", RFC 1305, March 1992. + + [29] Deutsch, P., "DEFLATE Compressed Data Format Specification + version 1.3", RFC 1951, May 1996. + + [30] S. Spero, "Analysis of HTTP Performance Problems," + http://sunsite.unc.edu/mdma-release/http-prob.html. + + [31] Deutsch, P. and J. Gailly, "ZLIB Compressed Data Format + Specification version 3.3", RFC 1950, May 1996. + + [32] Franks, J., Hallam-Baker, P., Hostetler, J., Leach, P., + Luotonen, A., Sink, E. and L. Stewart, "An Extension to HTTP: + Digest Access Authentication", RFC 2069, January 1997. + + [33] Fielding, R., Gettys, J., Mogul, J., Frystyk, H. and T. + Berners-Lee, "Hypertext Transfer Protocol -- HTTP/1.1", RFC + 2068, January 1997. + + [34] Bradner, S., "Key words for use in RFCs to Indicate Requirement + Levels", BCP 14, RFC 2119, March 1997. + + [35] Troost, R. and Dorner, S., "Communicating Presentation + Information in Internet Messages: The Content-Disposition + Header", RFC 1806, June 1995. + + [36] Mogul, J., Fielding, R., Gettys, J. and H. Frystyk, "Use and + Interpretation of HTTP Version Numbers", RFC 2145, May 1997. + [jg639] + + [37] Palme, J., "Common Internet Message Headers", RFC 2076, February + 1997. [jg640] + + + + + +Fielding, et al. Standards Track [Page 160] + +RFC 2616 HTTP/1.1 June 1999 + + + [38] Yergeau, F., "UTF-8, a transformation format of Unicode and + ISO-10646", RFC 2279, January 1998. [jg641] + + [39] Nielsen, H.F., Gettys, J., Baird-Smith, A., Prud'hommeaux, E., + Lie, H., and C. Lilley. "Network Performance Effects of + HTTP/1.1, CSS1, and PNG," Proceedings of ACM SIGCOMM '97, Cannes + France, September 1997.[jg642] + + [40] Freed, N. and N. Borenstein, "Multipurpose Internet Mail + Extensions (MIME) Part Two: Media Types", RFC 2046, November + 1996. [jg643] + + [41] Alvestrand, H., "IETF Policy on Character Sets and Languages", + BCP 18, RFC 2277, January 1998. [jg644] + + [42] Berners-Lee, T., Fielding, R. and L. Masinter, "Uniform Resource + Identifiers (URI): Generic Syntax and Semantics", RFC 2396, + August 1998. [jg645] + + [43] Franks, J., Hallam-Baker, P., Hostetler, J., Lawrence, S., + Leach, P., Luotonen, A., Sink, E. and L. Stewart, "HTTP + Authentication: Basic and Digest Access Authentication", RFC + 2617, June 1999. [jg646] + + [44] Luotonen, A., "Tunneling TCP based protocols through Web proxy + servers," Work in Progress. [jg647] + + [45] Palme, J. and A. Hopmann, "MIME E-mail Encapsulation of + Aggregate Documents, such as HTML (MHTML)", RFC 2110, March + 1997. + + [46] Bradner, S., "The Internet Standards Process -- Revision 3", BCP + 9, RFC 2026, October 1996. + + [47] Masinter, L., "Hyper Text Coffee Pot Control Protocol + (HTCPCP/1.0)", RFC 2324, 1 April 1998. + + [48] Freed, N. and N. Borenstein, "Multipurpose Internet Mail + Extensions (MIME) Part Five: Conformance Criteria and Examples", + RFC 2049, November 1996. + + [49] Troost, R., Dorner, S. and K. Moore, "Communicating Presentation + Information in Internet Messages: The Content-Disposition Header + Field", RFC 2183, August 1997. + + + + + + + +Fielding, et al. Standards Track [Page 161] + +RFC 2616 HTTP/1.1 June 1999 + + +18 Authors' Addresses + + Roy T. Fielding + Information and Computer Science + University of California, Irvine + Irvine, CA 92697-3425, USA + + Fax: +1 (949) 824-1715 + EMail: fielding@ics.uci.edu + + + James Gettys + World Wide Web Consortium + MIT Laboratory for Computer Science + 545 Technology Square + Cambridge, MA 02139, USA + + Fax: +1 (617) 258 8682 + EMail: jg@w3.org + + + Jeffrey C. Mogul + Western Research Laboratory + Compaq Computer Corporation + 250 University Avenue + Palo Alto, California, 94305, USA + + EMail: mogul@wrl.dec.com + + + Henrik Frystyk Nielsen + World Wide Web Consortium + MIT Laboratory for Computer Science + 545 Technology Square + Cambridge, MA 02139, USA + + Fax: +1 (617) 258 8682 + EMail: frystyk@w3.org + + + Larry Masinter + Xerox Corporation + 3333 Coyote Hill Road + Palo Alto, CA 94034, USA + + EMail: masinter@parc.xerox.com + + + + + +Fielding, et al. Standards Track [Page 162] + +RFC 2616 HTTP/1.1 June 1999 + + + Paul J. Leach + Microsoft Corporation + 1 Microsoft Way + Redmond, WA 98052, USA + + EMail: paulle@microsoft.com + + + Tim Berners-Lee + Director, World Wide Web Consortium + MIT Laboratory for Computer Science + 545 Technology Square + Cambridge, MA 02139, USA + + Fax: +1 (617) 258 8682 + EMail: timbl@w3.org + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +Fielding, et al. Standards Track [Page 163] + +RFC 2616 HTTP/1.1 June 1999 + + +19 Appendices + +19.1 Internet Media Type message/http and application/http + + In addition to defining the HTTP/1.1 protocol, this document serves + as the specification for the Internet media type "message/http" and + "application/http". The message/http type can be used to enclose a + single HTTP request or response message, provided that it obeys the + MIME restrictions for all "message" types regarding line length and + encodings. The application/http type can be used to enclose a + pipeline of one or more HTTP request or response messages (not + intermixed). The following is to be registered with IANA [17]. + + Media Type name: message + Media subtype name: http + Required parameters: none + Optional parameters: version, msgtype + version: The HTTP-Version number of the enclosed message + (e.g., "1.1"). If not present, the version can be + determined from the first line of the body. + msgtype: The message type -- "request" or "response". If not + present, the type can be determined from the first + line of the body. + Encoding considerations: only "7bit", "8bit", or "binary" are + permitted + Security considerations: none + + Media Type name: application + Media subtype name: http + Required parameters: none + Optional parameters: version, msgtype + version: The HTTP-Version number of the enclosed messages + (e.g., "1.1"). If not present, the version can be + determined from the first line of the body. + msgtype: The message type -- "request" or "response". If not + present, the type can be determined from the first + line of the body. + Encoding considerations: HTTP messages enclosed by this type + are in "binary" format; use of an appropriate + Content-Transfer-Encoding is required when + transmitted via E-mail. + Security considerations: none + + + + + + + + + +Fielding, et al. Standards Track [Page 164] + +RFC 2616 HTTP/1.1 June 1999 + + +19.2 Internet Media Type multipart/byteranges + + When an HTTP 206 (Partial Content) response message includes the + content of multiple ranges (a response to a request for multiple + non-overlapping ranges), these are transmitted as a multipart + message-body. The media type for this purpose is called + "multipart/byteranges". + + The multipart/byteranges media type includes two or more parts, each + with its own Content-Type and Content-Range fields. The required + boundary parameter specifies the boundary string used to separate + each body-part. + + Media Type name: multipart + Media subtype name: byteranges + Required parameters: boundary + Optional parameters: none + Encoding considerations: only "7bit", "8bit", or "binary" are + permitted + Security considerations: none + + + For example: + + HTTP/1.1 206 Partial Content + Date: Wed, 15 Nov 1995 06:25:24 GMT + Last-Modified: Wed, 15 Nov 1995 04:58:08 GMT + Content-type: multipart/byteranges; boundary=THIS_STRING_SEPARATES + + --THIS_STRING_SEPARATES + Content-type: application/pdf + Content-range: bytes 500-999/8000 + + ...the first range... + --THIS_STRING_SEPARATES + Content-type: application/pdf + Content-range: bytes 7000-7999/8000 + + ...the second range + --THIS_STRING_SEPARATES-- + + Notes: + + 1) Additional CRLFs may precede the first boundary string in the + entity. + + + + + + +Fielding, et al. Standards Track [Page 165] + +RFC 2616 HTTP/1.1 June 1999 + + + 2) Although RFC 2046 [40] permits the boundary string to be + quoted, some existing implementations handle a quoted boundary + string incorrectly. + + 3) A number of browsers and servers were coded to an early draft + of the byteranges specification to use a media type of + multipart/x-byteranges, which is almost, but not quite + compatible with the version documented in HTTP/1.1. + +19.3 Tolerant Applications + + Although this document specifies the requirements for the generation + of HTTP/1.1 messages, not all applications will be correct in their + implementation. We therefore recommend that operational applications + be tolerant of deviations whenever those deviations can be + interpreted unambiguously. + + Clients SHOULD be tolerant in parsing the Status-Line and servers + tolerant when parsing the Request-Line. In particular, they SHOULD + accept any amount of SP or HT characters between fields, even though + only a single SP is required. + + The line terminator for message-header fields is the sequence CRLF. + However, we recommend that applications, when parsing such headers, + recognize a single LF as a line terminator and ignore the leading CR. + + The character set of an entity-body SHOULD be labeled as the lowest + common denominator of the character codes used within that body, with + the exception that not labeling the entity is preferred over labeling + the entity with the labels US-ASCII or ISO-8859-1. See section 3.7.1 + and 3.4.1. + + Additional rules for requirements on parsing and encoding of dates + and other potential problems with date encodings include: + + - HTTP/1.1 clients and caches SHOULD assume that an RFC-850 date + which appears to be more than 50 years in the future is in fact + in the past (this helps solve the "year 2000" problem). + + - An HTTP/1.1 implementation MAY internally represent a parsed + Expires date as earlier than the proper value, but MUST NOT + internally represent a parsed Expires date as later than the + proper value. + + - All expiration-related calculations MUST be done in GMT. The + local time zone MUST NOT influence the calculation or comparison + of an age or expiration time. + + + + +Fielding, et al. Standards Track [Page 166] + +RFC 2616 HTTP/1.1 June 1999 + + + - If an HTTP header incorrectly carries a date value with a time + zone other than GMT, it MUST be converted into GMT using the + most conservative possible conversion. + +19.4 Differences Between HTTP Entities and RFC 2045 Entities + + HTTP/1.1 uses many of the constructs defined for Internet Mail (RFC + 822 [9]) and the Multipurpose Internet Mail Extensions (MIME [7]) to + allow entities to be transmitted in an open variety of + representations and with extensible mechanisms. However, RFC 2045 + discusses mail, and HTTP has a few features that are different from + those described in RFC 2045. These differences were carefully chosen + to optimize performance over binary connections, to allow greater + freedom in the use of new media types, to make date comparisons + easier, and to acknowledge the practice of some early HTTP servers + and clients. + + This appendix describes specific areas where HTTP differs from RFC + 2045. Proxies and gateways to strict MIME environments SHOULD be + aware of these differences and provide the appropriate conversions + where necessary. Proxies and gateways from MIME environments to HTTP + also need to be aware of the differences because some conversions + might be required. + +19.4.1 MIME-Version + + HTTP is not a MIME-compliant protocol. However, HTTP/1.1 messages MAY + include a single MIME-Version general-header field to indicate what + version of the MIME protocol was used to construct the message. Use + of the MIME-Version header field indicates that the message is in + full compliance with the MIME protocol (as defined in RFC 2045[7]). + Proxies/gateways are responsible for ensuring full compliance (where + possible) when exporting HTTP messages to strict MIME environments. + + MIME-Version = "MIME-Version" ":" 1*DIGIT "." 1*DIGIT + + MIME version "1.0" is the default for use in HTTP/1.1. However, + HTTP/1.1 message parsing and semantics are defined by this document + and not the MIME specification. + +19.4.2 Conversion to Canonical Form + + RFC 2045 [7] requires that an Internet mail entity be converted to + canonical form prior to being transferred, as described in section 4 + of RFC 2049 [48]. Section 3.7.1 of this document describes the forms + allowed for subtypes of the "text" media type when transmitted over + HTTP. RFC 2046 requires that content with a type of "text" represent + line breaks as CRLF and forbids the use of CR or LF outside of line + + + +Fielding, et al. Standards Track [Page 167] + +RFC 2616 HTTP/1.1 June 1999 + + + break sequences. HTTP allows CRLF, bare CR, and bare LF to indicate a + line break within text content when a message is transmitted over + HTTP. + + Where it is possible, a proxy or gateway from HTTP to a strict MIME + environment SHOULD translate all line breaks within the text media + types described in section 3.7.1 of this document to the RFC 2049 + canonical form of CRLF. Note, however, that this might be complicated + by the presence of a Content-Encoding and by the fact that HTTP + allows the use of some character sets which do not use octets 13 and + 10 to represent CR and LF, as is the case for some multi-byte + character sets. + + Implementors should note that conversion will break any cryptographic + checksums applied to the original content unless the original content + is already in canonical form. Therefore, the canonical form is + recommended for any content that uses such checksums in HTTP. + +19.4.3 Conversion of Date Formats + + HTTP/1.1 uses a restricted set of date formats (section 3.3.1) to + simplify the process of date comparison. Proxies and gateways from + other protocols SHOULD ensure that any Date header field present in a + message conforms to one of the HTTP/1.1 formats and rewrite the date + if necessary. + +19.4.4 Introduction of Content-Encoding + + RFC 2045 does not include any concept equivalent to HTTP/1.1's + Content-Encoding header field. Since this acts as a modifier on the + media type, proxies and gateways from HTTP to MIME-compliant + protocols MUST either change the value of the Content-Type header + field or decode the entity-body before forwarding the message. (Some + experimental applications of Content-Type for Internet mail have used + a media-type parameter of ";conversions=" to perform + a function equivalent to Content-Encoding. However, this parameter is + not part of RFC 2045.) + +19.4.5 No Content-Transfer-Encoding + + HTTP does not use the Content-Transfer-Encoding (CTE) field of RFC + 2045. Proxies and gateways from MIME-compliant protocols to HTTP MUST + remove any non-identity CTE ("quoted-printable" or "base64") encoding + prior to delivering the response message to an HTTP client. + + Proxies and gateways from HTTP to MIME-compliant protocols are + responsible for ensuring that the message is in the correct format + and encoding for safe transport on that protocol, where "safe + + + +Fielding, et al. Standards Track [Page 168] + +RFC 2616 HTTP/1.1 June 1999 + + + transport" is defined by the limitations of the protocol being used. + Such a proxy or gateway SHOULD label the data with an appropriate + Content-Transfer-Encoding if doing so will improve the likelihood of + safe transport over the destination protocol. + +19.4.6 Introduction of Transfer-Encoding + + HTTP/1.1 introduces the Transfer-Encoding header field (section + 14.41). Proxies/gateways MUST remove any transfer-coding prior to + forwarding a message via a MIME-compliant protocol. + + A process for decoding the "chunked" transfer-coding (section 3.6) + can be represented in pseudo-code as: + + length := 0 + read chunk-size, chunk-extension (if any) and CRLF + while (chunk-size > 0) { + read chunk-data and CRLF + append chunk-data to entity-body + length := length + chunk-size + read chunk-size and CRLF + } + read entity-header + while (entity-header not empty) { + append entity-header to existing header fields + read entity-header + } + Content-Length := length + Remove "chunked" from Transfer-Encoding + +19.4.7 MHTML and Line Length Limitations + + HTTP implementations which share code with MHTML [45] implementations + need to be aware of MIME line length limitations. Since HTTP does not + have this limitation, HTTP does not fold long lines. MHTML messages + being transported by HTTP follow all conventions of MHTML, including + line length limitations and folding, canonicalization, etc., since + HTTP transports all message-bodies as payload (see section 3.7.2) and + does not interpret the content or any MIME header lines that might be + contained therein. + +19.5 Additional Features + + RFC 1945 and RFC 2068 document protocol elements used by some + existing HTTP implementations, but not consistently and correctly + across most HTTP/1.1 applications. Implementors are advised to be + aware of these features, but cannot rely upon their presence in, or + interoperability with, other HTTP/1.1 applications. Some of these + + + +Fielding, et al. Standards Track [Page 169] + +RFC 2616 HTTP/1.1 June 1999 + + + describe proposed experimental features, and some describe features + that experimental deployment found lacking that are now addressed in + the base HTTP/1.1 specification. + + A number of other headers, such as Content-Disposition and Title, + from SMTP and MIME are also often implemented (see RFC 2076 [37]). + +19.5.1 Content-Disposition + + The Content-Disposition response-header field has been proposed as a + means for the origin server to suggest a default filename if the user + requests that the content is saved to a file. This usage is derived + from the definition of Content-Disposition in RFC 1806 [35]. + + content-disposition = "Content-Disposition" ":" + disposition-type *( ";" disposition-parm ) + disposition-type = "attachment" | disp-extension-token + disposition-parm = filename-parm | disp-extension-parm + filename-parm = "filename" "=" quoted-string + disp-extension-token = token + disp-extension-parm = token "=" ( token | quoted-string ) + + An example is + + Content-Disposition: attachment; filename="fname.ext" + + The receiving user agent SHOULD NOT respect any directory path + information present in the filename-parm parameter, which is the only + parameter believed to apply to HTTP implementations at this time. The + filename SHOULD be treated as a terminal component only. + + If this header is used in a response with the application/octet- + stream content-type, the implied suggestion is that the user agent + should not display the response, but directly enter a `save response + as...' dialog. + + See section 15.5 for Content-Disposition security issues. + +19.6 Compatibility with Previous Versions + + It is beyond the scope of a protocol specification to mandate + compliance with previous versions. HTTP/1.1 was deliberately + designed, however, to make supporting previous versions easy. It is + worth noting that, at the time of composing this specification + (1996), we would expect commercial HTTP/1.1 servers to: + + - recognize the format of the Request-Line for HTTP/0.9, 1.0, and + 1.1 requests; + + + +Fielding, et al. Standards Track [Page 170] + +RFC 2616 HTTP/1.1 June 1999 + + + - understand any valid request in the format of HTTP/0.9, 1.0, or + 1.1; + + - respond appropriately with a message in the same major version + used by the client. + + And we would expect HTTP/1.1 clients to: + + - recognize the format of the Status-Line for HTTP/1.0 and 1.1 + responses; + + - understand any valid response in the format of HTTP/0.9, 1.0, or + 1.1. + + For most implementations of HTTP/1.0, each connection is established + by the client prior to the request and closed by the server after + sending the response. Some implementations implement the Keep-Alive + version of persistent connections described in section 19.7.1 of RFC + 2068 [33]. + +19.6.1 Changes from HTTP/1.0 + + This section summarizes major differences between versions HTTP/1.0 + and HTTP/1.1. + +19.6.1.1 Changes to Simplify Multi-homed Web Servers and Conserve IP + Addresses + + The requirements that clients and servers support the Host request- + header, report an error if the Host request-header (section 14.23) is + missing from an HTTP/1.1 request, and accept absolute URIs (section + 5.1.2) are among the most important changes defined by this + specification. + + Older HTTP/1.0 clients assumed a one-to-one relationship of IP + addresses and servers; there was no other established mechanism for + distinguishing the intended server of a request than the IP address + to which that request was directed. The changes outlined above will + allow the Internet, once older HTTP clients are no longer common, to + support multiple Web sites from a single IP address, greatly + simplifying large operational Web servers, where allocation of many + IP addresses to a single host has created serious problems. The + Internet will also be able to recover the IP addresses that have been + allocated for the sole purpose of allowing special-purpose domain + names to be used in root-level HTTP URLs. Given the rate of growth of + the Web, and the number of servers already deployed, it is extremely + + + + + +Fielding, et al. Standards Track [Page 171] + +RFC 2616 HTTP/1.1 June 1999 + + + important that all implementations of HTTP (including updates to + existing HTTP/1.0 applications) correctly implement these + requirements: + + - Both clients and servers MUST support the Host request-header. + + - A client that sends an HTTP/1.1 request MUST send a Host header. + + - Servers MUST report a 400 (Bad Request) error if an HTTP/1.1 + request does not include a Host request-header. + + - Servers MUST accept absolute URIs. + +19.6.2 Compatibility with HTTP/1.0 Persistent Connections + + Some clients and servers might wish to be compatible with some + previous implementations of persistent connections in HTTP/1.0 + clients and servers. Persistent connections in HTTP/1.0 are + explicitly negotiated as they are not the default behavior. HTTP/1.0 + experimental implementations of persistent connections are faulty, + and the new facilities in HTTP/1.1 are designed to rectify these + problems. The problem was that some existing 1.0 clients may be + sending Keep-Alive to a proxy server that doesn't understand + Connection, which would then erroneously forward it to the next + inbound server, which would establish the Keep-Alive connection and + result in a hung HTTP/1.0 proxy waiting for the close on the + response. The result is that HTTP/1.0 clients must be prevented from + using Keep-Alive when talking to proxies. + + However, talking to proxies is the most important use of persistent + connections, so that prohibition is clearly unacceptable. Therefore, + we need some other mechanism for indicating a persistent connection + is desired, which is safe to use even when talking to an old proxy + that ignores Connection. Persistent connections are the default for + HTTP/1.1 messages; we introduce a new keyword (Connection: close) for + declaring non-persistence. See section 14.10. + + The original HTTP/1.0 form of persistent connections (the Connection: + Keep-Alive and Keep-Alive header) is documented in RFC 2068. [33] + +19.6.3 Changes from RFC 2068 + + This specification has been carefully audited to correct and + disambiguate key word usage; RFC 2068 had many problems in respect to + the conventions laid out in RFC 2119 [34]. + + Clarified which error code should be used for inbound server failures + (e.g. DNS failures). (Section 10.5.5). + + + +Fielding, et al. Standards Track [Page 172] + +RFC 2616 HTTP/1.1 June 1999 + + + CREATE had a race that required an Etag be sent when a resource is + first created. (Section 10.2.2). + + Content-Base was deleted from the specification: it was not + implemented widely, and there is no simple, safe way to introduce it + without a robust extension mechanism. In addition, it is used in a + similar, but not identical fashion in MHTML [45]. + + Transfer-coding and message lengths all interact in ways that + required fixing exactly when chunked encoding is used (to allow for + transfer encoding that may not be self delimiting); it was important + to straighten out exactly how message lengths are computed. (Sections + 3.6, 4.4, 7.2.2, 13.5.2, 14.13, 14.16) + + A content-coding of "identity" was introduced, to solve problems + discovered in caching. (section 3.5) + + Quality Values of zero should indicate that "I don't want something" + to allow clients to refuse a representation. (Section 3.9) + + The use and interpretation of HTTP version numbers has been clarified + by RFC 2145. Require proxies to upgrade requests to highest protocol + version they support to deal with problems discovered in HTTP/1.0 + implementations (Section 3.1) + + Charset wildcarding is introduced to avoid explosion of character set + names in accept headers. (Section 14.2) + + A case was missed in the Cache-Control model of HTTP/1.1; s-maxage + was introduced to add this missing case. (Sections 13.4, 14.8, 14.9, + 14.9.3) + + The Cache-Control: max-age directive was not properly defined for + responses. (Section 14.9.3) + + There are situations where a server (especially a proxy) does not + know the full length of a response but is capable of serving a + byterange request. We therefore need a mechanism to allow byteranges + with a content-range not indicating the full length of the message. + (Section 14.16) + + Range request responses would become very verbose if all meta-data + were always returned; by allowing the server to only send needed + headers in a 206 response, this problem can be avoided. (Section + 10.2.7, 13.5.3, and 14.27) + + + + + + +Fielding, et al. Standards Track [Page 173] + +RFC 2616 HTTP/1.1 June 1999 + + + Fix problem with unsatisfiable range requests; there are two cases: + syntactic problems, and range doesn't exist in the document. The 416 + status code was needed to resolve this ambiguity needed to indicate + an error for a byte range request that falls outside of the actual + contents of a document. (Section 10.4.17, 14.16) + + Rewrite of message transmission requirements to make it much harder + for implementors to get it wrong, as the consequences of errors here + can have significant impact on the Internet, and to deal with the + following problems: + + 1. Changing "HTTP/1.1 or later" to "HTTP/1.1", in contexts where + this was incorrectly placing a requirement on the behavior of + an implementation of a future version of HTTP/1.x + + 2. Made it clear that user-agents should retry requests, not + "clients" in general. + + 3. Converted requirements for clients to ignore unexpected 100 + (Continue) responses, and for proxies to forward 100 responses, + into a general requirement for 1xx responses. + + 4. Modified some TCP-specific language, to make it clearer that + non-TCP transports are possible for HTTP. + + 5. Require that the origin server MUST NOT wait for the request + body before it sends a required 100 (Continue) response. + + 6. Allow, rather than require, a server to omit 100 (Continue) if + it has already seen some of the request body. + + 7. Allow servers to defend against denial-of-service attacks and + broken clients. + + This change adds the Expect header and 417 status code. The message + transmission requirements fixes are in sections 8.2, 10.4.18, + 8.1.2.2, 13.11, and 14.20. + + Proxies should be able to add Content-Length when appropriate. + (Section 13.5.2) + + Clean up confusion between 403 and 404 responses. (Section 10.4.4, + 10.4.5, and 10.4.11) + + Warnings could be cached incorrectly, or not updated appropriately. + (Section 13.1.2, 13.2.4, 13.5.2, 13.5.3, 14.9.3, and 14.46) Warning + also needed to be a general header, as PUT or other methods may have + need for it in requests. + + + +Fielding, et al. Standards Track [Page 174] + +RFC 2616 HTTP/1.1 June 1999 + + + Transfer-coding had significant problems, particularly with + interactions with chunked encoding. The solution is that transfer- + codings become as full fledged as content-codings. This involves + adding an IANA registry for transfer-codings (separate from content + codings), a new header field (TE) and enabling trailer headers in the + future. Transfer encoding is a major performance benefit, so it was + worth fixing [39]. TE also solves another, obscure, downward + interoperability problem that could have occurred due to interactions + between authentication trailers, chunked encoding and HTTP/1.0 + clients.(Section 3.6, 3.6.1, and 14.39) + + The PATCH, LINK, UNLINK methods were defined but not commonly + implemented in previous versions of this specification. See RFC 2068 + [33]. + + The Alternates, Content-Version, Derived-From, Link, URI, Public and + Content-Base header fields were defined in previous versions of this + specification, but not commonly implemented. See RFC 2068 [33]. + +20 Index + + Please see the PostScript version of this RFC for the INDEX. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +Fielding, et al. Standards Track [Page 175] + +RFC 2616 HTTP/1.1 June 1999 + + +21. Full Copyright Statement + + Copyright (C) The Internet Society (1999). All Rights Reserved. + + This document and translations of it may be copied and furnished to + others, and derivative works that comment on or otherwise explain it + or assist in its implementation may be prepared, copied, published + and distributed, in whole or in part, without restriction of any + kind, provided that the above copyright notice and this paragraph are + included on all such copies and derivative works. However, this + document itself may not be modified in any way, such as by removing + the copyright notice or references to the Internet Society or other + Internet organizations, except as needed for the purpose of + developing Internet standards in which case the procedures for + copyrights defined in the Internet Standards process must be + followed, or as required to translate it into languages other than + English. + + The limited permissions granted above are perpetual and will not be + revoked by the Internet Society or its successors or assigns. + + This document and the information contained herein is provided on an + "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING + TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING + BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION + HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF + MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. + +Acknowledgement + + Funding for the RFC Editor function is currently provided by the + Internet Society. + + + + + + + + + + + + + + + + + + + +Fielding, et al. Standards Track [Page 176] + diff --git a/docs/rfc3986.txt b/docs/rfc3986.txt new file mode 100644 index 0000000..c56ed4e --- /dev/null +++ b/docs/rfc3986.txt @@ -0,0 +1,3419 @@ + + + + + + +Network Working Group T. Berners-Lee +Request for Comments: 3986 W3C/MIT +STD: 66 R. Fielding +Updates: 1738 Day Software +Obsoletes: 2732, 2396, 1808 L. Masinter +Category: Standards Track Adobe Systems + January 2005 + + + Uniform Resource Identifier (URI): Generic Syntax + +Status of This Memo + + This document specifies an Internet standards track protocol for the + Internet community, and requests discussion and suggestions for + improvements. Please refer to the current edition of the "Internet + Official Protocol Standards" (STD 1) for the standardization state + and status of this protocol. Distribution of this memo is unlimited. + +Copyright Notice + + Copyright (C) The Internet Society (2005). + +Abstract + + A Uniform Resource Identifier (URI) is a compact sequence of + characters that identifies an abstract or physical resource. This + specification defines the generic URI syntax and a process for + resolving URI references that might be in relative form, along with + guidelines and security considerations for the use of URIs on the + Internet. The URI syntax defines a grammar that is a superset of all + valid URIs, allowing an implementation to parse the common components + of a URI reference without knowing the scheme-specific requirements + of every possible identifier. This specification does not define a + generative grammar for URIs; that task is performed by the individual + specifications of each URI scheme. + + + + + + + + + + + + + + + +Berners-Lee, et al. Standards Track [Page 1] + +RFC 3986 URI Generic Syntax January 2005 + + +Table of Contents + + 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 + 1.1. Overview of URIs . . . . . . . . . . . . . . . . . . . . 4 + 1.1.1. Generic Syntax . . . . . . . . . . . . . . . . . 6 + 1.1.2. Examples . . . . . . . . . . . . . . . . . . . . 7 + 1.1.3. URI, URL, and URN . . . . . . . . . . . . . . . 7 + 1.2. Design Considerations . . . . . . . . . . . . . . . . . 8 + 1.2.1. Transcription . . . . . . . . . . . . . . . . . 8 + 1.2.2. Separating Identification from Interaction . . . 9 + 1.2.3. Hierarchical Identifiers . . . . . . . . . . . . 10 + 1.3. Syntax Notation . . . . . . . . . . . . . . . . . . . . 11 + 2. Characters . . . . . . . . . . . . . . . . . . . . . . . . . . 11 + 2.1. Percent-Encoding . . . . . . . . . . . . . . . . . . . . 12 + 2.2. Reserved Characters . . . . . . . . . . . . . . . . . . 12 + 2.3. Unreserved Characters . . . . . . . . . . . . . . . . . 13 + 2.4. When to Encode or Decode . . . . . . . . . . . . . . . . 14 + 2.5. Identifying Data . . . . . . . . . . . . . . . . . . . . 14 + 3. Syntax Components . . . . . . . . . . . . . . . . . . . . . . 16 + 3.1. Scheme . . . . . . . . . . . . . . . . . . . . . . . . . 17 + 3.2. Authority . . . . . . . . . . . . . . . . . . . . . . . 17 + 3.2.1. User Information . . . . . . . . . . . . . . . . 18 + 3.2.2. Host . . . . . . . . . . . . . . . . . . . . . . 18 + 3.2.3. Port . . . . . . . . . . . . . . . . . . . . . . 22 + 3.3. Path . . . . . . . . . . . . . . . . . . . . . . . . . . 22 + 3.4. Query . . . . . . . . . . . . . . . . . . . . . . . . . 23 + 3.5. Fragment . . . . . . . . . . . . . . . . . . . . . . . . 24 + 4. Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 + 4.1. URI Reference . . . . . . . . . . . . . . . . . . . . . 25 + 4.2. Relative Reference . . . . . . . . . . . . . . . . . . . 26 + 4.3. Absolute URI . . . . . . . . . . . . . . . . . . . . . . 27 + 4.4. Same-Document Reference . . . . . . . . . . . . . . . . 27 + 4.5. Suffix Reference . . . . . . . . . . . . . . . . . . . . 27 + 5. Reference Resolution . . . . . . . . . . . . . . . . . . . . . 28 + 5.1. Establishing a Base URI . . . . . . . . . . . . . . . . 28 + 5.1.1. Base URI Embedded in Content . . . . . . . . . . 29 + 5.1.2. Base URI from the Encapsulating Entity . . . . . 29 + 5.1.3. Base URI from the Retrieval URI . . . . . . . . 30 + 5.1.4. Default Base URI . . . . . . . . . . . . . . . . 30 + 5.2. Relative Resolution . . . . . . . . . . . . . . . . . . 30 + 5.2.1. Pre-parse the Base URI . . . . . . . . . . . . . 31 + 5.2.2. Transform References . . . . . . . . . . . . . . 31 + 5.2.3. Merge Paths . . . . . . . . . . . . . . . . . . 32 + 5.2.4. Remove Dot Segments . . . . . . . . . . . . . . 33 + 5.3. Component Recomposition . . . . . . . . . . . . . . . . 35 + 5.4. Reference Resolution Examples . . . . . . . . . . . . . 35 + 5.4.1. Normal Examples . . . . . . . . . . . . . . . . 36 + 5.4.2. Abnormal Examples . . . . . . . . . . . . . . . 36 + + + +Berners-Lee, et al. Standards Track [Page 2] + +RFC 3986 URI Generic Syntax January 2005 + + + 6. Normalization and Comparison . . . . . . . . . . . . . . . . . 38 + 6.1. Equivalence . . . . . . . . . . . . . . . . . . . . . . 38 + 6.2. Comparison Ladder . . . . . . . . . . . . . . . . . . . 39 + 6.2.1. Simple String Comparison . . . . . . . . . . . . 39 + 6.2.2. Syntax-Based Normalization . . . . . . . . . . . 40 + 6.2.3. Scheme-Based Normalization . . . . . . . . . . . 41 + 6.2.4. Protocol-Based Normalization . . . . . . . . . . 42 + 7. Security Considerations . . . . . . . . . . . . . . . . . . . 43 + 7.1. Reliability and Consistency . . . . . . . . . . . . . . 43 + 7.2. Malicious Construction . . . . . . . . . . . . . . . . . 43 + 7.3. Back-End Transcoding . . . . . . . . . . . . . . . . . . 44 + 7.4. Rare IP Address Formats . . . . . . . . . . . . . . . . 45 + 7.5. Sensitive Information . . . . . . . . . . . . . . . . . 45 + 7.6. Semantic Attacks . . . . . . . . . . . . . . . . . . . . 45 + 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 46 + 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 46 + 10. References . . . . . . . . . . . . . . . . . . . . . . . . . . 46 + 10.1. Normative References . . . . . . . . . . . . . . . . . . 46 + 10.2. Informative References . . . . . . . . . . . . . . . . . 47 + A. Collected ABNF for URI . . . . . . . . . . . . . . . . . . . . 49 + B. Parsing a URI Reference with a Regular Expression . . . . . . 50 + C. Delimiting a URI in Context . . . . . . . . . . . . . . . . . 51 + D. Changes from RFC 2396 . . . . . . . . . . . . . . . . . . . . 53 + D.1. Additions . . . . . . . . . . . . . . . . . . . . . . . 53 + D.2. Modifications . . . . . . . . . . . . . . . . . . . . . 53 + Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 + Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 60 + Full Copyright Statement . . . . . . . . . . . . . . . . . . . . . 61 + + + + + + + + + + + + + + + + + + + + + + + +Berners-Lee, et al. Standards Track [Page 3] + +RFC 3986 URI Generic Syntax January 2005 + + +1. Introduction + + A Uniform Resource Identifier (URI) provides a simple and extensible + means for identifying a resource. This specification of URI syntax + and semantics is derived from concepts introduced by the World Wide + Web global information initiative, whose use of these identifiers + dates from 1990 and is described in "Universal Resource Identifiers + in WWW" [RFC1630]. The syntax is designed to meet the + recommendations laid out in "Functional Recommendations for Internet + Resource Locators" [RFC1736] and "Functional Requirements for Uniform + Resource Names" [RFC1737]. + + This document obsoletes [RFC2396], which merged "Uniform Resource + Locators" [RFC1738] and "Relative Uniform Resource Locators" + [RFC1808] in order to define a single, generic syntax for all URIs. + It obsoletes [RFC2732], which introduced syntax for an IPv6 address. + It excludes portions of RFC 1738 that defined the specific syntax of + individual URI schemes; those portions will be updated as separate + documents. The process for registration of new URI schemes is + defined separately by [BCP35]. Advice for designers of new URI + schemes can be found in [RFC2718]. All significant changes from RFC + 2396 are noted in Appendix D. + + This specification uses the terms "character" and "coded character + set" in accordance with the definitions provided in [BCP19], and + "character encoding" in place of what [BCP19] refers to as a + "charset". + +1.1. Overview of URIs + + URIs are characterized as follows: + + Uniform + + Uniformity provides several benefits. It allows different types + of resource identifiers to be used in the same context, even when + the mechanisms used to access those resources may differ. It + allows uniform semantic interpretation of common syntactic + conventions across different types of resource identifiers. It + allows introduction of new types of resource identifiers without + interfering with the way that existing identifiers are used. It + allows the identifiers to be reused in many different contexts, + thus permitting new applications or protocols to leverage a pre- + existing, large, and widely used set of resource identifiers. + + + + + + + +Berners-Lee, et al. Standards Track [Page 4] + +RFC 3986 URI Generic Syntax January 2005 + + + Resource + + This specification does not limit the scope of what might be a + resource; rather, the term "resource" is used in a general sense + for whatever might be identified by a URI. Familiar examples + include an electronic document, an image, a source of information + with a consistent purpose (e.g., "today's weather report for Los + Angeles"), a service (e.g., an HTTP-to-SMS gateway), and a + collection of other resources. A resource is not necessarily + accessible via the Internet; e.g., human beings, corporations, and + bound books in a library can also be resources. Likewise, + abstract concepts can be resources, such as the operators and + operands of a mathematical equation, the types of a relationship + (e.g., "parent" or "employee"), or numeric values (e.g., zero, + one, and infinity). + + Identifier + + An identifier embodies the information required to distinguish + what is being identified from all other things within its scope of + identification. Our use of the terms "identify" and "identifying" + refer to this purpose of distinguishing one resource from all + other resources, regardless of how that purpose is accomplished + (e.g., by name, address, or context). These terms should not be + mistaken as an assumption that an identifier defines or embodies + the identity of what is referenced, though that may be the case + for some identifiers. Nor should it be assumed that a system + using URIs will access the resource identified: in many cases, + URIs are used to denote resources without any intention that they + be accessed. Likewise, the "one" resource identified might not be + singular in nature (e.g., a resource might be a named set or a + mapping that varies over time). + + A URI is an identifier consisting of a sequence of characters + matching the syntax rule named in Section 3. It enables + uniform identification of resources via a separately defined + extensible set of naming schemes (Section 3.1). How that + identification is accomplished, assigned, or enabled is delegated to + each scheme specification. + + This specification does not place any limits on the nature of a + resource, the reasons why an application might seek to refer to a + resource, or the kinds of systems that might use URIs for the sake of + identifying resources. This specification does not require that a + URI persists in identifying the same resource over time, though that + is a common goal of all URI schemes. Nevertheless, nothing in this + + + + + +Berners-Lee, et al. Standards Track [Page 5] + +RFC 3986 URI Generic Syntax January 2005 + + + specification prevents an application from limiting itself to + particular types of resources, or to a subset of URIs that maintains + characteristics desired by that application. + + URIs have a global scope and are interpreted consistently regardless + of context, though the result of that interpretation may be in + relation to the end-user's context. For example, "http://localhost/" + has the same interpretation for every user of that reference, even + though the network interface corresponding to "localhost" may be + different for each end-user: interpretation is independent of access. + However, an action made on the basis of that reference will take + place in relation to the end-user's context, which implies that an + action intended to refer to a globally unique thing must use a URI + that distinguishes that resource from all other things. URIs that + identify in relation to the end-user's local context should only be + used when the context itself is a defining aspect of the resource, + such as when an on-line help manual refers to a file on the end- + user's file system (e.g., "file:///etc/hosts"). + +1.1.1. Generic Syntax + + Each URI begins with a scheme name, as defined in Section 3.1, that + refers to a specification for assigning identifiers within that + scheme. As such, the URI syntax is a federated and extensible naming + system wherein each scheme's specification may further restrict the + syntax and semantics of identifiers using that scheme. + + This specification defines those elements of the URI syntax that are + required of all URI schemes or are common to many URI schemes. It + thus defines the syntax and semantics needed to implement a scheme- + independent parsing mechanism for URI references, by which the + scheme-dependent handling of a URI can be postponed until the + scheme-dependent semantics are needed. Likewise, protocols and data + formats that make use of URI references can refer to this + specification as a definition for the range of syntax allowed for all + URIs, including those schemes that have yet to be defined. This + decouples the evolution of identification schemes from the evolution + of protocols, data formats, and implementations that make use of + URIs. + + A parser of the generic URI syntax can parse any URI reference into + its major components. Once the scheme is determined, further + scheme-specific parsing can be performed on the components. In other + words, the URI generic syntax is a superset of the syntax of all URI + schemes. + + + + + + +Berners-Lee, et al. Standards Track [Page 6] + +RFC 3986 URI Generic Syntax January 2005 + + +1.1.2. Examples + + The following example URIs illustrate several URI schemes and + variations in their common syntax components: + + ftp://ftp.is.co.za/rfc/rfc1808.txt + + http://www.ietf.org/rfc/rfc2396.txt + + ldap://[2001:db8::7]/c=GB?objectClass?one + + mailto:John.Doe@example.com + + news:comp.infosystems.www.servers.unix + + tel:+1-816-555-1212 + + telnet://192.0.2.16:80/ + + urn:oasis:names:specification:docbook:dtd:xml:4.1.2 + + +1.1.3. URI, URL, and URN + + A URI can be further classified as a locator, a name, or both. The + term "Uniform Resource Locator" (URL) refers to the subset of URIs + that, in addition to identifying a resource, provide a means of + locating the resource by describing its primary access mechanism + (e.g., its network "location"). The term "Uniform Resource Name" + (URN) has been used historically to refer to both URIs under the + "urn" scheme [RFC2141], which are required to remain globally unique + and persistent even when the resource ceases to exist or becomes + unavailable, and to any other URI with the properties of a name. + + An individual scheme does not have to be classified as being just one + of "name" or "locator". Instances of URIs from any given scheme may + have the characteristics of names or locators or both, often + depending on the persistence and care in the assignment of + identifiers by the naming authority, rather than on any quality of + the scheme. Future specifications and related documentation should + use the general term "URI" rather than the more restrictive terms + "URL" and "URN" [RFC3305]. + + + + + + + + + +Berners-Lee, et al. Standards Track [Page 7] + +RFC 3986 URI Generic Syntax January 2005 + + +1.2. Design Considerations + +1.2.1. Transcription + + The URI syntax has been designed with global transcription as one of + its main considerations. A URI is a sequence of characters from a + very limited set: the letters of the basic Latin alphabet, digits, + and a few special characters. A URI may be represented in a variety + of ways; e.g., ink on paper, pixels on a screen, or a sequence of + character encoding octets. The interpretation of a URI depends only + on the characters used and not on how those characters are + represented in a network protocol. + + The goal of transcription can be described by a simple scenario. + Imagine two colleagues, Sam and Kim, sitting in a pub at an + international conference and exchanging research ideas. Sam asks Kim + for a location to get more information, so Kim writes the URI for the + research site on a napkin. Upon returning home, Sam takes out the + napkin and types the URI into a computer, which then retrieves the + information to which Kim referred. + + There are several design considerations revealed by the scenario: + + o A URI is a sequence of characters that is not always represented + as a sequence of octets. + + o A URI might be transcribed from a non-network source and thus + should consist of characters that are most likely able to be + entered into a computer, within the constraints imposed by + keyboards (and related input devices) across languages and + locales. + + o A URI often has to be remembered by people, and it is easier for + people to remember a URI when it consists of meaningful or + familiar components. + + These design considerations are not always in alignment. For + example, it is often the case that the most meaningful name for a URI + component would require characters that cannot be typed into some + systems. The ability to transcribe a resource identifier from one + medium to another has been considered more important than having a + URI consist of the most meaningful of components. + + In local or regional contexts and with improving technology, users + might benefit from being able to use a wider range of characters; + such use is not defined by this specification. Percent-encoded + octets (Section 2.1) may be used within a URI to represent characters + outside the range of the US-ASCII coded character set if this + + + +Berners-Lee, et al. Standards Track [Page 8] + +RFC 3986 URI Generic Syntax January 2005 + + + representation is allowed by the scheme or by the protocol element in + which the URI is referenced. Such a definition should specify the + character encoding used to map those characters to octets prior to + being percent-encoded for the URI. + +1.2.2. Separating Identification from Interaction + + A common misunderstanding of URIs is that they are only used to refer + to accessible resources. The URI itself only provides + identification; access to the resource is neither guaranteed nor + implied by the presence of a URI. Instead, any operation associated + with a URI reference is defined by the protocol element, data format + attribute, or natural language text in which it appears. + + Given a URI, a system may attempt to perform a variety of operations + on the resource, as might be characterized by words such as "access", + "update", "replace", or "find attributes". Such operations are + defined by the protocols that make use of URIs, not by this + specification. However, we do use a few general terms for describing + common operations on URIs. URI "resolution" is the process of + determining an access mechanism and the appropriate parameters + necessary to dereference a URI; this resolution may require several + iterations. To use that access mechanism to perform an action on the + URI's resource is to "dereference" the URI. + + When URIs are used within information retrieval systems to identify + sources of information, the most common form of URI dereference is + "retrieval": making use of a URI in order to retrieve a + representation of its associated resource. A "representation" is a + sequence of octets, along with representation metadata describing + those octets, that constitutes a record of the state of the resource + at the time when the representation is generated. Retrieval is + achieved by a process that might include using the URI as a cache key + to check for a locally cached representation, resolution of the URI + to determine an appropriate access mechanism (if any), and + dereference of the URI for the sake of applying a retrieval + operation. Depending on the protocols used to perform the retrieval, + additional information might be supplied about the resource (resource + metadata) and its relation to other resources. + + URI references in information retrieval systems are designed to be + late-binding: the result of an access is generally determined when it + is accessed and may vary over time or due to other aspects of the + interaction. These references are created in order to be used in the + future: what is being identified is not some specific result that was + obtained in the past, but rather some characteristic that is expected + to be true for future results. In such cases, the resource referred + to by the URI is actually a sameness of characteristics as observed + + + +Berners-Lee, et al. Standards Track [Page 9] + +RFC 3986 URI Generic Syntax January 2005 + + + over time, perhaps elucidated by additional comments or assertions + made by the resource provider. + + Although many URI schemes are named after protocols, this does not + imply that use of these URIs will result in access to the resource + via the named protocol. URIs are often used simply for the sake of + identification. Even when a URI is used to retrieve a representation + of a resource, that access might be through gateways, proxies, + caches, and name resolution services that are independent of the + protocol associated with the scheme name. The resolution of some + URIs may require the use of more than one protocol (e.g., both DNS + and HTTP are typically used to access an "http" URI's origin server + when a representation isn't found in a local cache). + +1.2.3. Hierarchical Identifiers + + The URI syntax is organized hierarchically, with components listed in + order of decreasing significance from left to right. For some URI + schemes, the visible hierarchy is limited to the scheme itself: + everything after the scheme component delimiter (":") is considered + opaque to URI processing. Other URI schemes make the hierarchy + explicit and visible to generic parsing algorithms. + + The generic syntax uses the slash ("/"), question mark ("?"), and + number sign ("#") characters to delimit components that are + significant to the generic parser's hierarchical interpretation of an + identifier. In addition to aiding the readability of such + identifiers through the consistent use of familiar syntax, this + uniform representation of hierarchy across naming schemes allows + scheme-independent references to be made relative to that hierarchy. + + It is often the case that a group or "tree" of documents has been + constructed to serve a common purpose, wherein the vast majority of + URI references in these documents point to resources within the tree + rather than outside it. Similarly, documents located at a particular + site are much more likely to refer to other resources at that site + than to resources at remote sites. Relative referencing of URIs + allows document trees to be partially independent of their location + and access scheme. For instance, it is possible for a single set of + hypertext documents to be simultaneously accessible and traversable + via each of the "file", "http", and "ftp" schemes if the documents + refer to each other with relative references. Furthermore, such + document trees can be moved, as a whole, without changing any of the + relative references. + + A relative reference (Section 4.2) refers to a resource by describing + the difference within a hierarchical name space between the reference + context and the target URI. The reference resolution algorithm, + + + +Berners-Lee, et al. Standards Track [Page 10] + +RFC 3986 URI Generic Syntax January 2005 + + + presented in Section 5, defines how such a reference is transformed + to the target URI. As relative references can only be used within + the context of a hierarchical URI, designers of new URI schemes + should use a syntax consistent with the generic syntax's hierarchical + components unless there are compelling reasons to forbid relative + referencing within that scheme. + + NOTE: Previous specifications used the terms "partial URI" and + "relative URI" to denote a relative reference to a URI. As some + readers misunderstood those terms to mean that relative URIs are a + subset of URIs rather than a method of referencing URIs, this + specification simply refers to them as relative references. + + All URI references are parsed by generic syntax parsers when used. + However, because hierarchical processing has no effect on an absolute + URI used in a reference unless it contains one or more dot-segments + (complete path segments of "." or "..", as described in Section 3.3), + URI scheme specifications can define opaque identifiers by + disallowing use of slash characters, question mark characters, and + the URIs "scheme:." and "scheme:..". + +1.3. Syntax Notation + + This specification uses the Augmented Backus-Naur Form (ABNF) + notation of [RFC2234], including the following core ABNF syntax rules + defined by that specification: ALPHA (letters), CR (carriage return), + DIGIT (decimal digits), DQUOTE (double quote), HEXDIG (hexadecimal + digits), LF (line feed), and SP (space). The complete URI syntax is + collected in Appendix A. + +2. Characters + + The URI syntax provides a method of encoding data, presumably for the + sake of identifying a resource, as a sequence of characters. The URI + characters are, in turn, frequently encoded as octets for transport + or presentation. This specification does not mandate any particular + character encoding for mapping between URI characters and the octets + used to store or transmit those characters. When a URI appears in a + protocol element, the character encoding is defined by that protocol; + without such a definition, a URI is assumed to be in the same + character encoding as the surrounding text. + + The ABNF notation defines its terminal values to be non-negative + integers (codepoints) based on the US-ASCII coded character set + [ASCII]. Because a URI is a sequence of characters, we must invert + that relation in order to understand the URI syntax. Therefore, the + + + + + +Berners-Lee, et al. Standards Track [Page 11] + +RFC 3986 URI Generic Syntax January 2005 + + + integer values used by the ABNF must be mapped back to their + corresponding characters via US-ASCII in order to complete the syntax + rules. + + A URI is composed from a limited set of characters consisting of + digits, letters, and a few graphic symbols. A reserved subset of + those characters may be used to delimit syntax components within a + URI while the remaining characters, including both the unreserved set + and those reserved characters not acting as delimiters, define each + component's identifying data. + +2.1. Percent-Encoding + + A percent-encoding mechanism is used to represent a data octet in a + component when that octet's corresponding character is outside the + allowed set or is being used as a delimiter of, or within, the + component. A percent-encoded octet is encoded as a character + triplet, consisting of the percent character "%" followed by the two + hexadecimal digits representing that octet's numeric value. For + example, "%20" is the percent-encoding for the binary octet + "00100000" (ABNF: %x20), which in US-ASCII corresponds to the space + character (SP). Section 2.4 describes when percent-encoding and + decoding is applied. + + pct-encoded = "%" HEXDIG HEXDIG + + The uppercase hexadecimal digits 'A' through 'F' are equivalent to + the lowercase digits 'a' through 'f', respectively. If two URIs + differ only in the case of hexadecimal digits used in percent-encoded + octets, they are equivalent. For consistency, URI producers and + normalizers should use uppercase hexadecimal digits for all percent- + encodings. + +2.2. Reserved Characters + + URIs include components and subcomponents that are delimited by + characters in the "reserved" set. These characters are called + "reserved" because they may (or may not) be defined as delimiters by + the generic syntax, by each scheme-specific syntax, or by the + implementation-specific syntax of a URI's dereferencing algorithm. + If data for a URI component would conflict with a reserved + character's purpose as a delimiter, then the conflicting data must be + percent-encoded before the URI is formed. + + + + + + + + +Berners-Lee, et al. Standards Track [Page 12] + +RFC 3986 URI Generic Syntax January 2005 + + + reserved = gen-delims / sub-delims + + gen-delims = ":" / "/" / "?" / "#" / "[" / "]" / "@" + + sub-delims = "!" / "$" / "&" / "'" / "(" / ")" + / "*" / "+" / "," / ";" / "=" + + The purpose of reserved characters is to provide a set of delimiting + characters that are distinguishable from other data within a URI. + URIs that differ in the replacement of a reserved character with its + corresponding percent-encoded octet are not equivalent. Percent- + encoding a reserved character, or decoding a percent-encoded octet + that corresponds to a reserved character, will change how the URI is + interpreted by most applications. Thus, characters in the reserved + set are protected from normalization and are therefore safe to be + used by scheme-specific and producer-specific algorithms for + delimiting data subcomponents within a URI. + + A subset of the reserved characters (gen-delims) is used as + delimiters of the generic URI components described in Section 3. A + component's ABNF syntax rule will not use the reserved or gen-delims + rule names directly; instead, each syntax rule lists the characters + allowed within that component (i.e., not delimiting it), and any of + those characters that are also in the reserved set are "reserved" for + use as subcomponent delimiters within the component. Only the most + common subcomponents are defined by this specification; other + subcomponents may be defined by a URI scheme's specification, or by + the implementation-specific syntax of a URI's dereferencing + algorithm, provided that such subcomponents are delimited by + characters in the reserved set allowed within that component. + + URI producing applications should percent-encode data octets that + correspond to characters in the reserved set unless these characters + are specifically allowed by the URI scheme to represent data in that + component. If a reserved character is found in a URI component and + no delimiting role is known for that character, then it must be + interpreted as representing the data octet corresponding to that + character's encoding in US-ASCII. + +2.3. Unreserved Characters + + Characters that are allowed in a URI but do not have a reserved + purpose are called unreserved. These include uppercase and lowercase + letters, decimal digits, hyphen, period, underscore, and tilde. + + unreserved = ALPHA / DIGIT / "-" / "." / "_" / "~" + + + + + +Berners-Lee, et al. Standards Track [Page 13] + +RFC 3986 URI Generic Syntax January 2005 + + + URIs that differ in the replacement of an unreserved character with + its corresponding percent-encoded US-ASCII octet are equivalent: they + identify the same resource. However, URI comparison implementations + do not always perform normalization prior to comparison (see Section + 6). For consistency, percent-encoded octets in the ranges of ALPHA + (%41-%5A and %61-%7A), DIGIT (%30-%39), hyphen (%2D), period (%2E), + underscore (%5F), or tilde (%7E) should not be created by URI + producers and, when found in a URI, should be decoded to their + corresponding unreserved characters by URI normalizers. + +2.4. When to Encode or Decode + + Under normal circumstances, the only time when octets within a URI + are percent-encoded is during the process of producing the URI from + its component parts. This is when an implementation determines which + of the reserved characters are to be used as subcomponent delimiters + and which can be safely used as data. Once produced, a URI is always + in its percent-encoded form. + + When a URI is dereferenced, the components and subcomponents + significant to the scheme-specific dereferencing process (if any) + must be parsed and separated before the percent-encoded octets within + those components can be safely decoded, as otherwise the data may be + mistaken for component delimiters. The only exception is for + percent-encoded octets corresponding to characters in the unreserved + set, which can be decoded at any time. For example, the octet + corresponding to the tilde ("~") character is often encoded as "%7E" + by older URI processing implementations; the "%7E" can be replaced by + "~" without changing its interpretation. + + Because the percent ("%") character serves as the indicator for + percent-encoded octets, it must be percent-encoded as "%25" for that + octet to be used as data within a URI. Implementations must not + percent-encode or decode the same string more than once, as decoding + an already decoded string might lead to misinterpreting a percent + data octet as the beginning of a percent-encoding, or vice versa in + the case of percent-encoding an already percent-encoded string. + +2.5. Identifying Data + + URI characters provide identifying data for each of the URI + components, serving as an external interface for identification + between systems. Although the presence and nature of the URI + production interface is hidden from clients that use its URIs (and is + thus beyond the scope of the interoperability requirements defined by + this specification), it is a frequent source of confusion and errors + in the interpretation of URI character issues. Implementers have to + be aware that there are multiple character encodings involved in the + + + +Berners-Lee, et al. Standards Track [Page 14] + +RFC 3986 URI Generic Syntax January 2005 + + + production and transmission of URIs: local name and data encoding, + public interface encoding, URI character encoding, data format + encoding, and protocol encoding. + + Local names, such as file system names, are stored with a local + character encoding. URI producing applications (e.g., origin + servers) will typically use the local encoding as the basis for + producing meaningful names. The URI producer will transform the + local encoding to one that is suitable for a public interface and + then transform the public interface encoding into the restricted set + of URI characters (reserved, unreserved, and percent-encodings). + Those characters are, in turn, encoded as octets to be used as a + reference within a data format (e.g., a document charset), and such + data formats are often subsequently encoded for transmission over + Internet protocols. + + For most systems, an unreserved character appearing within a URI + component is interpreted as representing the data octet corresponding + to that character's encoding in US-ASCII. Consumers of URIs assume + that the letter "X" corresponds to the octet "01011000", and even + when that assumption is incorrect, there is no harm in making it. A + system that internally provides identifiers in the form of a + different character encoding, such as EBCDIC, will generally perform + character translation of textual identifiers to UTF-8 [STD63] (or + some other superset of the US-ASCII character encoding) at an + internal interface, thereby providing more meaningful identifiers + than those resulting from simply percent-encoding the original + octets. + + For example, consider an information service that provides data, + stored locally using an EBCDIC-based file system, to clients on the + Internet through an HTTP server. When an author creates a file with + the name "Laguna Beach" on that file system, the "http" URI + corresponding to that resource is expected to contain the meaningful + string "Laguna%20Beach". If, however, that server produces URIs by + using an overly simplistic raw octet mapping, then the result would + be a URI containing "%D3%81%87%A4%95%81@%C2%85%81%83%88". An + internal transcoding interface fixes this problem by transcoding the + local name to a superset of US-ASCII prior to producing the URI. + Naturally, proper interpretation of an incoming URI on such an + interface requires that percent-encoded octets be decoded (e.g., + "%20" to SP) before the reverse transcoding is applied to obtain the + local name. + + In some cases, the internal interface between a URI component and the + identifying data that it has been crafted to represent is much less + direct than a character encoding translation. For example, portions + of a URI might reflect a query on non-ASCII data, or numeric + + + +Berners-Lee, et al. Standards Track [Page 15] + +RFC 3986 URI Generic Syntax January 2005 + + + coordinates on a map. Likewise, a URI scheme may define components + with additional encoding requirements that are applied prior to + forming the component and producing the URI. + + When a new URI scheme defines a component that represents textual + data consisting of characters from the Universal Character Set [UCS], + the data should first be encoded as octets according to the UTF-8 + character encoding [STD63]; then only those octets that do not + correspond to characters in the unreserved set should be percent- + encoded. For example, the character A would be represented as "A", + the character LATIN CAPITAL LETTER A WITH GRAVE would be represented + as "%C3%80", and the character KATAKANA LETTER A would be represented + as "%E3%82%A2". + +3. Syntax Components + + The generic URI syntax consists of a hierarchical sequence of + components referred to as the scheme, authority, path, query, and + fragment. + + URI = scheme ":" hier-part [ "?" query ] [ "#" fragment ] + + hier-part = "//" authority path-abempty + / path-absolute + / path-rootless + / path-empty + + The scheme and path components are required, though the path may be + empty (no characters). When authority is present, the path must + either be empty or begin with a slash ("/") character. When + authority is not present, the path cannot begin with two slash + characters ("//"). These restrictions result in five different ABNF + rules for a path (Section 3.3), only one of which will match any + given URI reference. + + The following are two example URIs and their component parts: + + foo://example.com:8042/over/there?name=ferret#nose + \_/ \______________/\_________/ \_________/ \__/ + | | | | | + scheme authority path query fragment + | _____________________|__ + / \ / \ + urn:example:animal:ferret:nose + + + + + + + +Berners-Lee, et al. Standards Track [Page 16] + +RFC 3986 URI Generic Syntax January 2005 + + +3.1. Scheme + + Each URI begins with a scheme name that refers to a specification for + assigning identifiers within that scheme. As such, the URI syntax is + a federated and extensible naming system wherein each scheme's + specification may further restrict the syntax and semantics of + identifiers using that scheme. + + Scheme names consist of a sequence of characters beginning with a + letter and followed by any combination of letters, digits, plus + ("+"), period ("."), or hyphen ("-"). Although schemes are case- + insensitive, the canonical form is lowercase and documents that + specify schemes must do so with lowercase letters. An implementation + should accept uppercase letters as equivalent to lowercase in scheme + names (e.g., allow "HTTP" as well as "http") for the sake of + robustness but should only produce lowercase scheme names for + consistency. + + scheme = ALPHA *( ALPHA / DIGIT / "+" / "-" / "." ) + + Individual schemes are not specified by this document. The process + for registration of new URI schemes is defined separately by [BCP35]. + The scheme registry maintains the mapping between scheme names and + their specifications. Advice for designers of new URI schemes can be + found in [RFC2718]. URI scheme specifications must define their own + syntax so that all strings matching their scheme-specific syntax will + also match the grammar, as described in Section 4.3. + + When presented with a URI that violates one or more scheme-specific + restrictions, the scheme-specific resolution process should flag the + reference as an error rather than ignore the unused parts; doing so + reduces the number of equivalent URIs and helps detect abuses of the + generic syntax, which might indicate that the URI has been + constructed to mislead the user (Section 7.6). + +3.2. Authority + + Many URI schemes include a hierarchical element for a naming + authority so that governance of the name space defined by the + remainder of the URI is delegated to that authority (which may, in + turn, delegate it further). The generic syntax provides a common + means for distinguishing an authority based on a registered name or + server address, along with optional port and user information. + + The authority component is preceded by a double slash ("//") and is + terminated by the next slash ("/"), question mark ("?"), or number + sign ("#") character, or by the end of the URI. + + + + +Berners-Lee, et al. Standards Track [Page 17] + +RFC 3986 URI Generic Syntax January 2005 + + + authority = [ userinfo "@" ] host [ ":" port ] + + URI producers and normalizers should omit the ":" delimiter that + separates host from port if the port component is empty. Some + schemes do not allow the userinfo and/or port subcomponents. + + If a URI contains an authority component, then the path component + must either be empty or begin with a slash ("/") character. Non- + validating parsers (those that merely separate a URI reference into + its major components) will often ignore the subcomponent structure of + authority, treating it as an opaque string from the double-slash to + the first terminating delimiter, until such time as the URI is + dereferenced. + +3.2.1. User Information + + The userinfo subcomponent may consist of a user name and, optionally, + scheme-specific information about how to gain authorization to access + the resource. The user information, if present, is followed by a + commercial at-sign ("@") that delimits it from the host. + + userinfo = *( unreserved / pct-encoded / sub-delims / ":" ) + + Use of the format "user:password" in the userinfo field is + deprecated. Applications should not render as clear text any data + after the first colon (":") character found within a userinfo + subcomponent unless the data after the colon is the empty string + (indicating no password). Applications may choose to ignore or + reject such data when it is received as part of a reference and + should reject the storage of such data in unencrypted form. The + passing of authentication information in clear text has proven to be + a security risk in almost every case where it has been used. + + Applications that render a URI for the sake of user feedback, such as + in graphical hypertext browsing, should render userinfo in a way that + is distinguished from the rest of a URI, when feasible. Such + rendering will assist the user in cases where the userinfo has been + misleadingly crafted to look like a trusted domain name + (Section 7.6). + +3.2.2. Host + + The host subcomponent of authority is identified by an IP literal + encapsulated within square brackets, an IPv4 address in dotted- + decimal form, or a registered name. The host subcomponent is case- + insensitive. The presence of a host subcomponent within a URI does + not imply that the scheme requires access to the given host on the + Internet. In many cases, the host syntax is used only for the sake + + + +Berners-Lee, et al. Standards Track [Page 18] + +RFC 3986 URI Generic Syntax January 2005 + + + of reusing the existing registration process created and deployed for + DNS, thus obtaining a globally unique name without the cost of + deploying another registry. However, such use comes with its own + costs: domain name ownership may change over time for reasons not + anticipated by the URI producer. In other cases, the data within the + host component identifies a registered name that has nothing to do + with an Internet host. We use the name "host" for the ABNF rule + because that is its most common purpose, not its only purpose. + + host = IP-literal / IPv4address / reg-name + + The syntax rule for host is ambiguous because it does not completely + distinguish between an IPv4address and a reg-name. In order to + disambiguate the syntax, we apply the "first-match-wins" algorithm: + If host matches the rule for IPv4address, then it should be + considered an IPv4 address literal and not a reg-name. Although host + is case-insensitive, producers and normalizers should use lowercase + for registered names and hexadecimal addresses for the sake of + uniformity, while only using uppercase letters for percent-encodings. + + A host identified by an Internet Protocol literal address, version 6 + [RFC3513] or later, is distinguished by enclosing the IP literal + within square brackets ("[" and "]"). This is the only place where + square bracket characters are allowed in the URI syntax. In + anticipation of future, as-yet-undefined IP literal address formats, + an implementation may use an optional version flag to indicate such a + format explicitly rather than rely on heuristic determination. + + IP-literal = "[" ( IPv6address / IPvFuture ) "]" + + IPvFuture = "v" 1*HEXDIG "." 1*( unreserved / sub-delims / ":" ) + + The version flag does not indicate the IP version; rather, it + indicates future versions of the literal format. As such, + implementations must not provide the version flag for the existing + IPv4 and IPv6 literal address forms described below. If a URI + containing an IP-literal that starts with "v" (case-insensitive), + indicating that the version flag is present, is dereferenced by an + application that does not know the meaning of that version flag, then + the application should return an appropriate error for "address + mechanism not supported". + + A host identified by an IPv6 literal address is represented inside + the square brackets without a preceding version flag. The ABNF + provided here is a translation of the text definition of an IPv6 + literal address provided in [RFC3513]. This syntax does not support + IPv6 scoped addressing zone identifiers. + + + + +Berners-Lee, et al. Standards Track [Page 19] + +RFC 3986 URI Generic Syntax January 2005 + + + A 128-bit IPv6 address is divided into eight 16-bit pieces. Each + piece is represented numerically in case-insensitive hexadecimal, + using one to four hexadecimal digits (leading zeroes are permitted). + The eight encoded pieces are given most-significant first, separated + by colon characters. Optionally, the least-significant two pieces + may instead be represented in IPv4 address textual format. A + sequence of one or more consecutive zero-valued 16-bit pieces within + the address may be elided, omitting all their digits and leaving + exactly two consecutive colons in their place to mark the elision. + + IPv6address = 6( h16 ":" ) ls32 + / "::" 5( h16 ":" ) ls32 + / [ h16 ] "::" 4( h16 ":" ) ls32 + / [ *1( h16 ":" ) h16 ] "::" 3( h16 ":" ) ls32 + / [ *2( h16 ":" ) h16 ] "::" 2( h16 ":" ) ls32 + / [ *3( h16 ":" ) h16 ] "::" h16 ":" ls32 + / [ *4( h16 ":" ) h16 ] "::" ls32 + / [ *5( h16 ":" ) h16 ] "::" h16 + / [ *6( h16 ":" ) h16 ] "::" + + ls32 = ( h16 ":" h16 ) / IPv4address + ; least-significant 32 bits of address + + h16 = 1*4HEXDIG + ; 16 bits of address represented in hexadecimal + + A host identified by an IPv4 literal address is represented in + dotted-decimal notation (a sequence of four decimal numbers in the + range 0 to 255, separated by "."), as described in [RFC1123] by + reference to [RFC0952]. Note that other forms of dotted notation may + be interpreted on some platforms, as described in Section 7.4, but + only the dotted-decimal form of four octets is allowed by this + grammar. + + IPv4address = dec-octet "." dec-octet "." dec-octet "." dec-octet + + dec-octet = DIGIT ; 0-9 + / %x31-39 DIGIT ; 10-99 + / "1" 2DIGIT ; 100-199 + / "2" %x30-34 DIGIT ; 200-249 + / "25" %x30-35 ; 250-255 + + A host identified by a registered name is a sequence of characters + usually intended for lookup within a locally defined host or service + name registry, though the URI's scheme-specific semantics may require + that a specific registry (or fixed name table) be used instead. The + most common name registry mechanism is the Domain Name System (DNS). + A registered name intended for lookup in the DNS uses the syntax + + + +Berners-Lee, et al. Standards Track [Page 20] + +RFC 3986 URI Generic Syntax January 2005 + + + defined in Section 3.5 of [RFC1034] and Section 2.1 of [RFC1123]. + Such a name consists of a sequence of domain labels separated by ".", + each domain label starting and ending with an alphanumeric character + and possibly also containing "-" characters. The rightmost domain + label of a fully qualified domain name in DNS may be followed by a + single "." and should be if it is necessary to distinguish between + the complete domain name and some local domain. + + reg-name = *( unreserved / pct-encoded / sub-delims ) + + If the URI scheme defines a default for host, then that default + applies when the host subcomponent is undefined or when the + registered name is empty (zero length). For example, the "file" URI + scheme is defined so that no authority, an empty host, and + "localhost" all mean the end-user's machine, whereas the "http" + scheme considers a missing authority or empty host invalid. + + This specification does not mandate a particular registered name + lookup technology and therefore does not restrict the syntax of reg- + name beyond what is necessary for interoperability. Instead, it + delegates the issue of registered name syntax conformance to the + operating system of each application performing URI resolution, and + that operating system decides what it will allow for the purpose of + host identification. A URI resolution implementation might use DNS, + host tables, yellow pages, NetInfo, WINS, or any other system for + lookup of registered names. However, a globally scoped naming + system, such as DNS fully qualified domain names, is necessary for + URIs intended to have global scope. URI producers should use names + that conform to the DNS syntax, even when use of DNS is not + immediately apparent, and should limit these names to no more than + 255 characters in length. + + The reg-name syntax allows percent-encoded octets in order to + represent non-ASCII registered names in a uniform way that is + independent of the underlying name resolution technology. Non-ASCII + characters must first be encoded according to UTF-8 [STD63], and then + each octet of the corresponding UTF-8 sequence must be percent- + encoded to be represented as URI characters. URI producing + applications must not use percent-encoding in host unless it is used + to represent a UTF-8 character sequence. When a non-ASCII registered + name represents an internationalized domain name intended for + resolution via the DNS, the name must be transformed to the IDNA + encoding [RFC3490] prior to name lookup. URI producers should + provide these registered names in the IDNA encoding, rather than a + percent-encoding, if they wish to maximize interoperability with + legacy URI resolvers. + + + + + +Berners-Lee, et al. Standards Track [Page 21] + +RFC 3986 URI Generic Syntax January 2005 + + +3.2.3. Port + + The port subcomponent of authority is designated by an optional port + number in decimal following the host and delimited from it by a + single colon (":") character. + + port = *DIGIT + + A scheme may define a default port. For example, the "http" scheme + defines a default port of "80", corresponding to its reserved TCP + port number. The type of port designated by the port number (e.g., + TCP, UDP, SCTP) is defined by the URI scheme. URI producers and + normalizers should omit the port component and its ":" delimiter if + port is empty or if its value would be the same as that of the + scheme's default. + +3.3. Path + + The path component contains data, usually organized in hierarchical + form, that, along with data in the non-hierarchical query component + (Section 3.4), serves to identify a resource within the scope of the + URI's scheme and naming authority (if any). The path is terminated + by the first question mark ("?") or number sign ("#") character, or + by the end of the URI. + + If a URI contains an authority component, then the path component + must either be empty or begin with a slash ("/") character. If a URI + does not contain an authority component, then the path cannot begin + with two slash characters ("//"). In addition, a URI reference + (Section 4.1) may be a relative-path reference, in which case the + first path segment cannot contain a colon (":") character. The ABNF + requires five separate rules to disambiguate these cases, only one of + which will match the path substring within a given URI reference. We + use the generic term "path component" to describe the URI substring + matched by the parser to one of these rules. + + path = path-abempty ; begins with "/" or is empty + / path-absolute ; begins with "/" but not "//" + / path-noscheme ; begins with a non-colon segment + / path-rootless ; begins with a segment + / path-empty ; zero characters + + path-abempty = *( "/" segment ) + path-absolute = "/" [ segment-nz *( "/" segment ) ] + path-noscheme = segment-nz-nc *( "/" segment ) + path-rootless = segment-nz *( "/" segment ) + path-empty = 0 + + + + +Berners-Lee, et al. Standards Track [Page 22] + +RFC 3986 URI Generic Syntax January 2005 + + + segment = *pchar + segment-nz = 1*pchar + segment-nz-nc = 1*( unreserved / pct-encoded / sub-delims / "@" ) + ; non-zero-length segment without any colon ":" + + pchar = unreserved / pct-encoded / sub-delims / ":" / "@" + + A path consists of a sequence of path segments separated by a slash + ("/") character. A path is always defined for a URI, though the + defined path may be empty (zero length). Use of the slash character + to indicate hierarchy is only required when a URI will be used as the + context for relative references. For example, the URI + has a path of "fred@example.com", whereas + the URI has an empty path. + + The path segments "." and "..", also known as dot-segments, are + defined for relative reference within the path name hierarchy. They + are intended for use at the beginning of a relative-path reference + (Section 4.2) to indicate relative position within the hierarchical + tree of names. This is similar to their role within some operating + systems' file directory structures to indicate the current directory + and parent directory, respectively. However, unlike in a file + system, these dot-segments are only interpreted within the URI path + hierarchy and are removed as part of the resolution process (Section + 5.2). + + Aside from dot-segments in hierarchical paths, a path segment is + considered opaque by the generic syntax. URI producing applications + often use the reserved characters allowed in a segment to delimit + scheme-specific or dereference-handler-specific subcomponents. For + example, the semicolon (";") and equals ("=") reserved characters are + often used to delimit parameters and parameter values applicable to + that segment. The comma (",") reserved character is often used for + similar purposes. For example, one URI producer might use a segment + such as "name;v=1.1" to indicate a reference to version 1.1 of + "name", whereas another might use a segment such as "name,1.1" to + indicate the same. Parameter types may be defined by scheme-specific + semantics, but in most cases the syntax of a parameter is specific to + the implementation of the URI's dereferencing algorithm. + +3.4. Query + + The query component contains non-hierarchical data that, along with + data in the path component (Section 3.3), serves to identify a + resource within the scope of the URI's scheme and naming authority + (if any). The query component is indicated by the first question + mark ("?") character and terminated by a number sign ("#") character + or by the end of the URI. + + + +Berners-Lee, et al. Standards Track [Page 23] + +RFC 3986 URI Generic Syntax January 2005 + + + query = *( pchar / "/" / "?" ) + + The characters slash ("/") and question mark ("?") may represent data + within the query component. Beware that some older, erroneous + implementations may not handle such data correctly when it is used as + the base URI for relative references (Section 5.1), apparently + because they fail to distinguish query data from path data when + looking for hierarchical separators. However, as query components + are often used to carry identifying information in the form of + "key=value" pairs and one frequently used value is a reference to + another URI, it is sometimes better for usability to avoid percent- + encoding those characters. + +3.5. Fragment + + The fragment identifier component of a URI allows indirect + identification of a secondary resource by reference to a primary + resource and additional identifying information. The identified + secondary resource may be some portion or subset of the primary + resource, some view on representations of the primary resource, or + some other resource defined or described by those representations. A + fragment identifier component is indicated by the presence of a + number sign ("#") character and terminated by the end of the URI. + + fragment = *( pchar / "/" / "?" ) + + The semantics of a fragment identifier are defined by the set of + representations that might result from a retrieval action on the + primary resource. The fragment's format and resolution is therefore + dependent on the media type [RFC2046] of a potentially retrieved + representation, even though such a retrieval is only performed if the + URI is dereferenced. If no such representation exists, then the + semantics of the fragment are considered unknown and are effectively + unconstrained. Fragment identifier semantics are independent of the + URI scheme and thus cannot be redefined by scheme specifications. + + Individual media types may define their own restrictions on or + structures within the fragment identifier syntax for specifying + different types of subsets, views, or external references that are + identifiable as secondary resources by that media type. If the + primary resource has multiple representations, as is often the case + for resources whose representation is selected based on attributes of + the retrieval request (a.k.a., content negotiation), then whatever is + identified by the fragment should be consistent across all of those + representations. Each representation should either define the + fragment so that it corresponds to the same secondary resource, + regardless of how it is represented, or should leave the fragment + undefined (i.e., not found). + + + +Berners-Lee, et al. Standards Track [Page 24] + +RFC 3986 URI Generic Syntax January 2005 + + + As with any URI, use of a fragment identifier component does not + imply that a retrieval action will take place. A URI with a fragment + identifier may be used to refer to the secondary resource without any + implication that the primary resource is accessible or will ever be + accessed. + + Fragment identifiers have a special role in information retrieval + systems as the primary form of client-side indirect referencing, + allowing an author to specifically identify aspects of an existing + resource that are only indirectly provided by the resource owner. As + such, the fragment identifier is not used in the scheme-specific + processing of a URI; instead, the fragment identifier is separated + from the rest of the URI prior to a dereference, and thus the + identifying information within the fragment itself is dereferenced + solely by the user agent, regardless of the URI scheme. Although + this separate handling is often perceived to be a loss of + information, particularly for accurate redirection of references as + resources move over time, it also serves to prevent information + providers from denying reference authors the right to refer to + information within a resource selectively. Indirect referencing also + provides additional flexibility and extensibility to systems that use + URIs, as new media types are easier to define and deploy than new + schemes of identification. + + The characters slash ("/") and question mark ("?") are allowed to + represent data within the fragment identifier. Beware that some + older, erroneous implementations may not handle this data correctly + when it is used as the base URI for relative references (Section + 5.1). + +4. Usage + + When applications make reference to a URI, they do not always use the + full form of reference defined by the "URI" syntax rule. To save + space and take advantage of hierarchical locality, many Internet + protocol elements and media type formats allow an abbreviation of a + URI, whereas others restrict the syntax to a particular form of URI. + We define the most common forms of reference syntax in this + specification because they impact and depend upon the design of the + generic syntax, requiring a uniform parsing algorithm in order to be + interpreted consistently. + +4.1. URI Reference + + URI-reference is used to denote the most common usage of a resource + identifier. + + URI-reference = URI / relative-ref + + + +Berners-Lee, et al. Standards Track [Page 25] + +RFC 3986 URI Generic Syntax January 2005 + + + A URI-reference is either a URI or a relative reference. If the + URI-reference's prefix does not match the syntax of a scheme followed + by its colon separator, then the URI-reference is a relative + reference. + + A URI-reference is typically parsed first into the five URI + components, in order to determine what components are present and + whether the reference is relative. Then, each component is parsed + for its subparts and their validation. The ABNF of URI-reference, + along with the "first-match-wins" disambiguation rule, is sufficient + to define a validating parser for the generic syntax. Readers + familiar with regular expressions should see Appendix B for an + example of a non-validating URI-reference parser that will take any + given string and extract the URI components. + +4.2. Relative Reference + + A relative reference takes advantage of the hierarchical syntax + (Section 1.2.3) to express a URI reference relative to the name space + of another hierarchical URI. + + relative-ref = relative-part [ "?" query ] [ "#" fragment ] + + relative-part = "//" authority path-abempty + / path-absolute + / path-noscheme + / path-empty + + The URI referred to by a relative reference, also known as the target + URI, is obtained by applying the reference resolution algorithm of + Section 5. + + A relative reference that begins with two slash characters is termed + a network-path reference; such references are rarely used. A + relative reference that begins with a single slash character is + termed an absolute-path reference. A relative reference that does + not begin with a slash character is termed a relative-path reference. + + A path segment that contains a colon character (e.g., "this:that") + cannot be used as the first segment of a relative-path reference, as + it would be mistaken for a scheme name. Such a segment must be + preceded by a dot-segment (e.g., "./this:that") to make a relative- + path reference. + + + + + + + + +Berners-Lee, et al. Standards Track [Page 26] + +RFC 3986 URI Generic Syntax January 2005 + + +4.3. Absolute URI + + Some protocol elements allow only the absolute form of a URI without + a fragment identifier. For example, defining a base URI for later + use by relative references calls for an absolute-URI syntax rule that + does not allow a fragment. + + absolute-URI = scheme ":" hier-part [ "?" query ] + + URI scheme specifications must define their own syntax so that all + strings matching their scheme-specific syntax will also match the + grammar. Scheme specifications will not define + fragment identifier syntax or usage, regardless of its applicability + to resources identifiable via that scheme, as fragment identification + is orthogonal to scheme definition. However, scheme specifications + are encouraged to include a wide range of examples, including + examples that show use of the scheme's URIs with fragment identifiers + when such usage is appropriate. + +4.4. Same-Document Reference + + When a URI reference refers to a URI that is, aside from its fragment + component (if any), identical to the base URI (Section 5.1), that + reference is called a "same-document" reference. The most frequent + examples of same-document references are relative references that are + empty or include only the number sign ("#") separator followed by a + fragment identifier. + + When a same-document reference is dereferenced for a retrieval + action, the target of that reference is defined to be within the same + entity (representation, document, or message) as the reference; + therefore, a dereference should not result in a new retrieval action. + + Normalization of the base and target URIs prior to their comparison, + as described in Sections 6.2.2 and 6.2.3, is allowed but rarely + performed in practice. Normalization may increase the set of same- + document references, which may be of benefit to some caching + applications. As such, reference authors should not assume that a + slightly different, though equivalent, reference URI will (or will + not) be interpreted as a same-document reference by any given + application. + +4.5. Suffix Reference + + The URI syntax is designed for unambiguous reference to resources and + extensibility via the URI scheme. However, as URI identification and + usage have become commonplace, traditional media (television, radio, + newspapers, billboards, etc.) have increasingly used a suffix of the + + + +Berners-Lee, et al. Standards Track [Page 27] + +RFC 3986 URI Generic Syntax January 2005 + + + URI as a reference, consisting of only the authority and path + portions of the URI, such as + + www.w3.org/Addressing/ + + or simply a DNS registered name on its own. Such references are + primarily intended for human interpretation rather than for machines, + with the assumption that context-based heuristics are sufficient to + complete the URI (e.g., most registered names beginning with "www" + are likely to have a URI prefix of "http://"). Although there is no + standard set of heuristics for disambiguating a URI suffix, many + client implementations allow them to be entered by the user and + heuristically resolved. + + Although this practice of using suffix references is common, it + should be avoided whenever possible and should never be used in + situations where long-term references are expected. The heuristics + noted above will change over time, particularly when a new URI scheme + becomes popular, and are often incorrect when used out of context. + Furthermore, they can lead to security issues along the lines of + those described in [RFC1535]. + + As a URI suffix has the same syntax as a relative-path reference, a + suffix reference cannot be used in contexts where a relative + reference is expected. As a result, suffix references are limited to + places where there is no defined base URI, such as dialog boxes and + off-line advertisements. + +5. Reference Resolution + + This section defines the process of resolving a URI reference within + a context that allows relative references so that the result is a + string matching the syntax rule of Section 3. + +5.1. Establishing a Base URI + + The term "relative" implies that a "base URI" exists against which + the relative reference is applied. Aside from fragment-only + references (Section 4.4), relative references are only usable when a + base URI is known. A base URI must be established by the parser + prior to parsing URI references that might be relative. A base URI + must conform to the syntax rule (Section 4.3). If the + base URI is obtained from a URI reference, then that reference must + be converted to absolute form and stripped of any fragment component + prior to its use as a base URI. + + + + + + +Berners-Lee, et al. Standards Track [Page 28] + +RFC 3986 URI Generic Syntax January 2005 + + + The base URI of a reference can be established in one of four ways, + discussed below in order of precedence. The order of precedence can + be thought of in terms of layers, where the innermost defined base + URI has the highest precedence. This can be visualized graphically + as follows: + + .----------------------------------------------------------. + | .----------------------------------------------------. | + | | .----------------------------------------------. | | + | | | .----------------------------------------. | | | + | | | | .----------------------------------. | | | | + | | | | | | | | | | + | | | | `----------------------------------' | | | | + | | | | (5.1.1) Base URI embedded in content | | | | + | | | `----------------------------------------' | | | + | | | (5.1.2) Base URI of the encapsulating entity | | | + | | | (message, representation, or none) | | | + | | `----------------------------------------------' | | + | | (5.1.3) URI used to retrieve the entity | | + | `----------------------------------------------------' | + | (5.1.4) Default Base URI (application-dependent) | + `----------------------------------------------------------' + +5.1.1. Base URI Embedded in Content + + Within certain media types, a base URI for relative references can be + embedded within the content itself so that it can be readily obtained + by a parser. This can be useful for descriptive documents, such as + tables of contents, which may be transmitted to others through + protocols other than their usual retrieval context (e.g., email or + USENET news). + + It is beyond the scope of this specification to specify how, for each + media type, a base URI can be embedded. The appropriate syntax, when + available, is described by the data format specification associated + with each media type. + +5.1.2. Base URI from the Encapsulating Entity + + If no base URI is embedded, the base URI is defined by the + representation's retrieval context. For a document that is enclosed + within another entity, such as a message or archive, the retrieval + context is that entity. Thus, the default base URI of a + representation is the base URI of the entity in which the + representation is encapsulated. + + + + + + +Berners-Lee, et al. Standards Track [Page 29] + +RFC 3986 URI Generic Syntax January 2005 + + + A mechanism for embedding a base URI within MIME container types + (e.g., the message and multipart types) is defined by MHTML + [RFC2557]. Protocols that do not use the MIME message header syntax, + but that do allow some form of tagged metadata to be included within + messages, may define their own syntax for defining a base URI as part + of a message. + +5.1.3. Base URI from the Retrieval URI + + If no base URI is embedded and the representation is not encapsulated + within some other entity, then, if a URI was used to retrieve the + representation, that URI shall be considered the base URI. Note that + if the retrieval was the result of a redirected request, the last URI + used (i.e., the URI that resulted in the actual retrieval of the + representation) is the base URI. + +5.1.4. Default Base URI + + If none of the conditions described above apply, then the base URI is + defined by the context of the application. As this definition is + necessarily application-dependent, failing to define a base URI by + using one of the other methods may result in the same content being + interpreted differently by different types of applications. + + A sender of a representation containing relative references is + responsible for ensuring that a base URI for those references can be + established. Aside from fragment-only references, relative + references can only be used reliably in situations where the base URI + is well defined. + +5.2. Relative Resolution + + This section describes an algorithm for converting a URI reference + that might be relative to a given base URI into the parsed components + of the reference's target. The components can then be recomposed, as + described in Section 5.3, to form the target URI. This algorithm + provides definitive results that can be used to test the output of + other implementations. Applications may implement relative reference + resolution by using some other algorithm, provided that the results + match what would be given by this one. + + + + + + + + + + + +Berners-Lee, et al. Standards Track [Page 30] + +RFC 3986 URI Generic Syntax January 2005 + + +5.2.1. Pre-parse the Base URI + + The base URI (Base) is established according to the procedure of + Section 5.1 and parsed into the five main components described in + Section 3. Note that only the scheme component is required to be + present in a base URI; the other components may be empty or + undefined. A component is undefined if its associated delimiter does + not appear in the URI reference; the path component is never + undefined, though it may be empty. + + Normalization of the base URI, as described in Sections 6.2.2 and + 6.2.3, is optional. A URI reference must be transformed to its + target URI before it can be normalized. + +5.2.2. Transform References + + For each URI reference (R), the following pseudocode describes an + algorithm for transforming R into its target URI (T): + + -- The URI reference is parsed into the five URI components + -- + (R.scheme, R.authority, R.path, R.query, R.fragment) = parse(R); + + -- A non-strict parser may ignore a scheme in the reference + -- if it is identical to the base URI's scheme. + -- + if ((not strict) and (R.scheme == Base.scheme)) then + undefine(R.scheme); + endif; + + + + + + + + + + + + + + + + + + + + + + +Berners-Lee, et al. Standards Track [Page 31] + +RFC 3986 URI Generic Syntax January 2005 + + + if defined(R.scheme) then + T.scheme = R.scheme; + T.authority = R.authority; + T.path = remove_dot_segments(R.path); + T.query = R.query; + else + if defined(R.authority) then + T.authority = R.authority; + T.path = remove_dot_segments(R.path); + T.query = R.query; + else + if (R.path == "") then + T.path = Base.path; + if defined(R.query) then + T.query = R.query; + else + T.query = Base.query; + endif; + else + if (R.path starts-with "/") then + T.path = remove_dot_segments(R.path); + else + T.path = merge(Base.path, R.path); + T.path = remove_dot_segments(T.path); + endif; + T.query = R.query; + endif; + T.authority = Base.authority; + endif; + T.scheme = Base.scheme; + endif; + + T.fragment = R.fragment; + +5.2.3. Merge Paths + + The pseudocode above refers to a "merge" routine for merging a + relative-path reference with the path of the base URI. This is + accomplished as follows: + + o If the base URI has a defined authority component and an empty + path, then return a string consisting of "/" concatenated with the + reference's path; otherwise, + + + + + + + + +Berners-Lee, et al. Standards Track [Page 32] + +RFC 3986 URI Generic Syntax January 2005 + + + o return a string consisting of the reference's path component + appended to all but the last segment of the base URI's path (i.e., + excluding any characters after the right-most "/" in the base URI + path, or excluding the entire base URI path if it does not contain + any "/" characters). + +5.2.4. Remove Dot Segments + + The pseudocode also refers to a "remove_dot_segments" routine for + interpreting and removing the special "." and ".." complete path + segments from a referenced path. This is done after the path is + extracted from a reference, whether or not the path was relative, in + order to remove any invalid or extraneous dot-segments prior to + forming the target URI. Although there are many ways to accomplish + this removal process, we describe a simple method using two string + buffers. + + 1. The input buffer is initialized with the now-appended path + components and the output buffer is initialized to the empty + string. + + 2. While the input buffer is not empty, loop as follows: + + A. If the input buffer begins with a prefix of "../" or "./", + then remove that prefix from the input buffer; otherwise, + + B. if the input buffer begins with a prefix of "/./" or "/.", + where "." is a complete path segment, then replace that + prefix with "/" in the input buffer; otherwise, + + C. if the input buffer begins with a prefix of "/../" or "/..", + where ".." is a complete path segment, then replace that + prefix with "/" in the input buffer and remove the last + segment and its preceding "/" (if any) from the output + buffer; otherwise, + + D. if the input buffer consists only of "." or "..", then remove + that from the input buffer; otherwise, + + E. move the first path segment in the input buffer to the end of + the output buffer, including the initial "/" character (if + any) and any subsequent characters up to, but not including, + the next "/" character or the end of the input buffer. + + 3. Finally, the output buffer is returned as the result of + remove_dot_segments. + + + + + +Berners-Lee, et al. Standards Track [Page 33] + +RFC 3986 URI Generic Syntax January 2005 + + + Note that dot-segments are intended for use in URI references to + express an identifier relative to the hierarchy of names in the base + URI. The remove_dot_segments algorithm respects that hierarchy by + removing extra dot-segments rather than treat them as an error or + leaving them to be misinterpreted by dereference implementations. + + The following illustrates how the above steps are applied for two + examples of merged paths, showing the state of the two buffers after + each step. + + STEP OUTPUT BUFFER INPUT BUFFER + + 1 : /a/b/c/./../../g + 2E: /a /b/c/./../../g + 2E: /a/b /c/./../../g + 2E: /a/b/c /./../../g + 2B: /a/b/c /../../g + 2C: /a/b /../g + 2C: /a /g + 2E: /a/g + + STEP OUTPUT BUFFER INPUT BUFFER + + 1 : mid/content=5/../6 + 2E: mid /content=5/../6 + 2E: mid/content=5 /../6 + 2C: mid /6 + 2E: mid/6 + + Some applications may find it more efficient to implement the + remove_dot_segments algorithm by using two segment stacks rather than + strings. + + Note: Beware that some older, erroneous implementations will fail + to separate a reference's query component from its path component + prior to merging the base and reference paths, resulting in an + interoperability failure if the query component contains the + strings "/../" or "/./". + + + + + + + + + + + + + +Berners-Lee, et al. Standards Track [Page 34] + +RFC 3986 URI Generic Syntax January 2005 + + +5.3. Component Recomposition + + Parsed URI components can be recomposed to obtain the corresponding + URI reference string. Using pseudocode, this would be: + + result = "" + + if defined(scheme) then + append scheme to result; + append ":" to result; + endif; + + if defined(authority) then + append "//" to result; + append authority to result; + endif; + + append path to result; + + if defined(query) then + append "?" to result; + append query to result; + endif; + + if defined(fragment) then + append "#" to result; + append fragment to result; + endif; + + return result; + + Note that we are careful to preserve the distinction between a + component that is undefined, meaning that its separator was not + present in the reference, and a component that is empty, meaning that + the separator was present and was immediately followed by the next + component separator or the end of the reference. + +5.4. Reference Resolution Examples + + Within a representation with a well defined base URI of + + http://a/b/c/d;p?q + + a relative reference is transformed to its target URI as follows. + + + + + + + +Berners-Lee, et al. Standards Track [Page 35] + +RFC 3986 URI Generic Syntax January 2005 + + +5.4.1. Normal Examples + + "g:h" = "g:h" + "g" = "http://a/b/c/g" + "./g" = "http://a/b/c/g" + "g/" = "http://a/b/c/g/" + "/g" = "http://a/g" + "//g" = "http://g" + "?y" = "http://a/b/c/d;p?y" + "g?y" = "http://a/b/c/g?y" + "#s" = "http://a/b/c/d;p?q#s" + "g#s" = "http://a/b/c/g#s" + "g?y#s" = "http://a/b/c/g?y#s" + ";x" = "http://a/b/c/;x" + "g;x" = "http://a/b/c/g;x" + "g;x?y#s" = "http://a/b/c/g;x?y#s" + "" = "http://a/b/c/d;p?q" + "." = "http://a/b/c/" + "./" = "http://a/b/c/" + ".." = "http://a/b/" + "../" = "http://a/b/" + "../g" = "http://a/b/g" + "../.." = "http://a/" + "../../" = "http://a/" + "../../g" = "http://a/g" + +5.4.2. Abnormal Examples + + Although the following abnormal examples are unlikely to occur in + normal practice, all URI parsers should be capable of resolving them + consistently. Each example uses the same base as that above. + + Parsers must be careful in handling cases where there are more ".." + segments in a relative-path reference than there are hierarchical + levels in the base URI's path. Note that the ".." syntax cannot be + used to change the authority component of a URI. + + "../../../g" = "http://a/g" + "../../../../g" = "http://a/g" + + + + + + + + + + + + +Berners-Lee, et al. Standards Track [Page 36] + +RFC 3986 URI Generic Syntax January 2005 + + + Similarly, parsers must remove the dot-segments "." and ".." when + they are complete components of a path, but not when they are only + part of a segment. + + "/./g" = "http://a/g" + "/../g" = "http://a/g" + "g." = "http://a/b/c/g." + ".g" = "http://a/b/c/.g" + "g.." = "http://a/b/c/g.." + "..g" = "http://a/b/c/..g" + + Less likely are cases where the relative reference uses unnecessary + or nonsensical forms of the "." and ".." complete path segments. + + "./../g" = "http://a/b/g" + "./g/." = "http://a/b/c/g/" + "g/./h" = "http://a/b/c/g/h" + "g/../h" = "http://a/b/c/h" + "g;x=1/./y" = "http://a/b/c/g;x=1/y" + "g;x=1/../y" = "http://a/b/c/y" + + Some applications fail to separate the reference's query and/or + fragment components from the path component before merging it with + the base path and removing dot-segments. This error is rarely + noticed, as typical usage of a fragment never includes the hierarchy + ("/") character and the query component is not normally used within + relative references. + + "g?y/./x" = "http://a/b/c/g?y/./x" + "g?y/../x" = "http://a/b/c/g?y/../x" + "g#s/./x" = "http://a/b/c/g#s/./x" + "g#s/../x" = "http://a/b/c/g#s/../x" + + Some parsers allow the scheme name to be present in a relative + reference if it is the same as the base URI scheme. This is + considered to be a loophole in prior specifications of partial URI + [RFC1630]. Its use should be avoided but is allowed for backward + compatibility. + + "http:g" = "http:g" ; for strict parsers + / "http://a/b/c/g" ; for backward compatibility + + + + + + + + + + +Berners-Lee, et al. Standards Track [Page 37] + +RFC 3986 URI Generic Syntax January 2005 + + +6. Normalization and Comparison + + One of the most common operations on URIs is simple comparison: + determining whether two URIs are equivalent without using the URIs to + access their respective resource(s). A comparison is performed every + time a response cache is accessed, a browser checks its history to + color a link, or an XML parser processes tags within a namespace. + Extensive normalization prior to comparison of URIs is often used by + spiders and indexing engines to prune a search space or to reduce + duplication of request actions and response storage. + + URI comparison is performed for some particular purpose. Protocols + or implementations that compare URIs for different purposes will + often be subject to differing design trade-offs in regards to how + much effort should be spent in reducing aliased identifiers. This + section describes various methods that may be used to compare URIs, + the trade-offs between them, and the types of applications that might + use them. + +6.1. Equivalence + + Because URIs exist to identify resources, presumably they should be + considered equivalent when they identify the same resource. However, + this definition of equivalence is not of much practical use, as there + is no way for an implementation to compare two resources unless it + has full knowledge or control of them. For this reason, + determination of equivalence or difference of URIs is based on string + comparison, perhaps augmented by reference to additional rules + provided by URI scheme definitions. We use the terms "different" and + "equivalent" to describe the possible outcomes of such comparisons, + but there are many application-dependent versions of equivalence. + + Even though it is possible to determine that two URIs are equivalent, + URI comparison is not sufficient to determine whether two URIs + identify different resources. For example, an owner of two different + domain names could decide to serve the same resource from both, + resulting in two different URIs. Therefore, comparison methods are + designed to minimize false negatives while strictly avoiding false + positives. + + In testing for equivalence, applications should not directly compare + relative references; the references should be converted to their + respective target URIs before comparison. When URIs are compared to + select (or avoid) a network action, such as retrieval of a + representation, fragment components (if any) should be excluded from + the comparison. + + + + + +Berners-Lee, et al. Standards Track [Page 38] + +RFC 3986 URI Generic Syntax January 2005 + + +6.2. Comparison Ladder + + A variety of methods are used in practice to test URI equivalence. + These methods fall into a range, distinguished by the amount of + processing required and the degree to which the probability of false + negatives is reduced. As noted above, false negatives cannot be + eliminated. In practice, their probability can be reduced, but this + reduction requires more processing and is not cost-effective for all + applications. + + If this range of comparison practices is considered as a ladder, the + following discussion will climb the ladder, starting with practices + that are cheap but have a relatively higher chance of producing false + negatives, and proceeding to those that have higher computational + cost and lower risk of false negatives. + +6.2.1. Simple String Comparison + + If two URIs, when considered as character strings, are identical, + then it is safe to conclude that they are equivalent. This type of + equivalence test has very low computational cost and is in wide use + in a variety of applications, particularly in the domain of parsing. + + Testing strings for equivalence requires some basic precautions. + This procedure is often referred to as "bit-for-bit" or + "byte-for-byte" comparison, which is potentially misleading. Testing + strings for equality is normally based on pair comparison of the + characters that make up the strings, starting from the first and + proceeding until both strings are exhausted and all characters are + found to be equal, until a pair of characters compares unequal, or + until one of the strings is exhausted before the other. + + This character comparison requires that each pair of characters be + put in comparable form. For example, should one URI be stored in a + byte array in EBCDIC encoding and the second in a Java String object + (UTF-16), bit-for-bit comparisons applied naively will produce + errors. It is better to speak of equality on a character-for- + character basis rather than on a byte-for-byte or bit-for-bit basis. + In practical terms, character-by-character comparisons should be done + codepoint-by-codepoint after conversion to a common character + encoding. + + False negatives are caused by the production and use of URI aliases. + Unnecessary aliases can be reduced, regardless of the comparison + method, by consistently providing URI references in an already- + normalized form (i.e., a form identical to what would be produced + after normalization is applied, as described below). + + + + +Berners-Lee, et al. Standards Track [Page 39] + +RFC 3986 URI Generic Syntax January 2005 + + + Protocols and data formats often limit some URI comparisons to simple + string comparison, based on the theory that people and + implementations will, in their own best interest, be consistent in + providing URI references, or at least consistent enough to negate any + efficiency that might be obtained from further normalization. + +6.2.2. Syntax-Based Normalization + + Implementations may use logic based on the definitions provided by + this specification to reduce the probability of false negatives. + This processing is moderately higher in cost than character-for- + character string comparison. For example, an application using this + approach could reasonably consider the following two URIs equivalent: + + example://a/b/c/%7Bfoo%7D + eXAMPLE://a/./b/../b/%63/%7bfoo%7d + + Web user agents, such as browsers, typically apply this type of URI + normalization when determining whether a cached response is + available. Syntax-based normalization includes such techniques as + case normalization, percent-encoding normalization, and removal of + dot-segments. + +6.2.2.1. Case Normalization + + For all URIs, the hexadecimal digits within a percent-encoding + triplet (e.g., "%3a" versus "%3A") are case-insensitive and therefore + should be normalized to use uppercase letters for the digits A-F. + + When a URI uses components of the generic syntax, the component + syntax equivalence rules always apply; namely, that the scheme and + host are case-insensitive and therefore should be normalized to + lowercase. For example, the URI is + equivalent to . The other generic syntax + components are assumed to be case-sensitive unless specifically + defined otherwise by the scheme (see Section 6.2.3). + +6.2.2.2. Percent-Encoding Normalization + + The percent-encoding mechanism (Section 2.1) is a frequent source of + variance among otherwise identical URIs. In addition to the case + normalization issue noted above, some URI producers percent-encode + octets that do not require percent-encoding, resulting in URIs that + are equivalent to their non-encoded counterparts. These URIs should + be normalized by decoding any percent-encoded octet that corresponds + to an unreserved character, as described in Section 2.3. + + + + + +Berners-Lee, et al. Standards Track [Page 40] + +RFC 3986 URI Generic Syntax January 2005 + + +6.2.2.3. Path Segment Normalization + + The complete path segments "." and ".." are intended only for use + within relative references (Section 4.1) and are removed as part of + the reference resolution process (Section 5.2). However, some + deployed implementations incorrectly assume that reference resolution + is not necessary when the reference is already a URI and thus fail to + remove dot-segments when they occur in non-relative paths. URI + normalizers should remove dot-segments by applying the + remove_dot_segments algorithm to the path, as described in + Section 5.2.4. + +6.2.3. Scheme-Based Normalization + + The syntax and semantics of URIs vary from scheme to scheme, as + described by the defining specification for each scheme. + Implementations may use scheme-specific rules, at further processing + cost, to reduce the probability of false negatives. For example, + because the "http" scheme makes use of an authority component, has a + default port of "80", and defines an empty path to be equivalent to + "/", the following four URIs are equivalent: + + http://example.com + http://example.com/ + http://example.com:/ + http://example.com:80/ + + In general, a URI that uses the generic syntax for authority with an + empty path should be normalized to a path of "/". Likewise, an + explicit ":port", for which the port is empty or the default for the + scheme, is equivalent to one where the port and its ":" delimiter are + elided and thus should be removed by scheme-based normalization. For + example, the second URI above is the normal form for the "http" + scheme. + + Another case where normalization varies by scheme is in the handling + of an empty authority component or empty host subcomponent. For many + scheme specifications, an empty authority or host is considered an + error; for others, it is considered equivalent to "localhost" or the + end-user's host. When a scheme defines a default for authority and a + URI reference to that default is desired, the reference should be + normalized to an empty authority for the sake of uniformity, brevity, + and internationalization. If, however, either the userinfo or port + subcomponents are non-empty, then the host should be given explicitly + even if it matches the default. + + Normalization should not remove delimiters when their associated + component is empty unless licensed to do so by the scheme + + + +Berners-Lee, et al. Standards Track [Page 41] + +RFC 3986 URI Generic Syntax January 2005 + + + specification. For example, the URI "http://example.com/?" cannot be + assumed to be equivalent to any of the examples above. Likewise, the + presence or absence of delimiters within a userinfo subcomponent is + usually significant to its interpretation. The fragment component is + not subject to any scheme-based normalization; thus, two URIs that + differ only by the suffix "#" are considered different regardless of + the scheme. + + Some schemes define additional subcomponents that consist of case- + insensitive data, giving an implicit license to normalizers to + convert this data to a common case (e.g., all lowercase). For + example, URI schemes that define a subcomponent of path to contain an + Internet hostname, such as the "mailto" URI scheme, cause that + subcomponent to be case-insensitive and thus subject to case + normalization (e.g., "mailto:Joe@Example.COM" is equivalent to + "mailto:Joe@example.com", even though the generic syntax considers + the path component to be case-sensitive). + + Other scheme-specific normalizations are possible. + +6.2.4. Protocol-Based Normalization + + Substantial effort to reduce the incidence of false negatives is + often cost-effective for web spiders. Therefore, they implement even + more aggressive techniques in URI comparison. For example, if they + observe that a URI such as + + http://example.com/data + + redirects to a URI differing only in the trailing slash + + http://example.com/data/ + + they will likely regard the two as equivalent in the future. This + kind of technique is only appropriate when equivalence is clearly + indicated by both the result of accessing the resources and the + common conventions of their scheme's dereference algorithm (in this + case, use of redirection by HTTP origin servers to avoid problems + with relative references). + + + + + + + + + + + + +Berners-Lee, et al. Standards Track [Page 42] + +RFC 3986 URI Generic Syntax January 2005 + + +7. Security Considerations + + A URI does not in itself pose a security threat. However, as URIs + are often used to provide a compact set of instructions for access to + network resources, care must be taken to properly interpret the data + within a URI, to prevent that data from causing unintended access, + and to avoid including data that should not be revealed in plain + text. + +7.1. Reliability and Consistency + + There is no guarantee that once a URI has been used to retrieve + information, the same information will be retrievable by that URI in + the future. Nor is there any guarantee that the information + retrievable via that URI in the future will be observably similar to + that retrieved in the past. The URI syntax does not constrain how a + given scheme or authority apportions its namespace or maintains it + over time. Such guarantees can only be obtained from the person(s) + controlling that namespace and the resource in question. A specific + URI scheme may define additional semantics, such as name persistence, + if those semantics are required of all naming authorities for that + scheme. + +7.2. Malicious Construction + + It is sometimes possible to construct a URI so that an attempt to + perform a seemingly harmless, idempotent operation, such as the + retrieval of a representation, will in fact cause a possibly damaging + remote operation. The unsafe URI is typically constructed by + specifying a port number other than that reserved for the network + protocol in question. The client unwittingly contacts a site running + a different protocol service, and data within the URI contains + instructions that, when interpreted according to this other protocol, + cause an unexpected operation. A frequent example of such abuse has + been the use of a protocol-based scheme with a port component of + "25", thereby fooling user agent software into sending an unintended + or impersonating message via an SMTP server. + + Applications should prevent dereference of a URI that specifies a TCP + port number within the "well-known port" range (0 - 1023) unless the + protocol being used to dereference that URI is compatible with the + protocol expected on that well-known port. Although IANA maintains a + registry of well-known ports, applications should make such + restrictions user-configurable to avoid preventing the deployment of + new services. + + + + + + +Berners-Lee, et al. Standards Track [Page 43] + +RFC 3986 URI Generic Syntax January 2005 + + + When a URI contains percent-encoded octets that match the delimiters + for a given resolution or dereference protocol (for example, CR and + LF characters for the TELNET protocol), these percent-encodings must + not be decoded before transmission across that protocol. Transfer of + the percent-encoding, which might violate the protocol, is less + harmful than allowing decoded octets to be interpreted as additional + operations or parameters, perhaps triggering an unexpected and + possibly harmful remote operation. + +7.3. Back-End Transcoding + + When a URI is dereferenced, the data within it is often parsed by + both the user agent and one or more servers. In HTTP, for example, a + typical user agent will parse a URI into its five major components, + access the authority's server, and send it the data within the + authority, path, and query components. A typical server will take + that information, parse the path into segments and the query into + key/value pairs, and then invoke implementation-specific handlers to + respond to the request. As a result, a common security concern for + server implementations that handle a URI, either as a whole or split + into separate components, is proper interpretation of the octet data + represented by the characters and percent-encodings within that URI. + + Percent-encoded octets must be decoded at some point during the + dereference process. Applications must split the URI into its + components and subcomponents prior to decoding the octets, as + otherwise the decoded octets might be mistaken for delimiters. + Security checks of the data within a URI should be applied after + decoding the octets. Note, however, that the "%00" percent-encoding + (NUL) may require special handling and should be rejected if the + application is not expecting to receive raw data within a component. + + Special care should be taken when the URI path interpretation process + involves the use of a back-end file system or related system + functions. File systems typically assign an operational meaning to + special characters, such as the "/", "\", ":", "[", and "]" + characters, and to special device names like ".", "..", "...", "aux", + "lpt", etc. In some cases, merely testing for the existence of such + a name will cause the operating system to pause or invoke unrelated + system calls, leading to significant security concerns regarding + denial of service and unintended data transfer. It would be + impossible for this specification to list all such significant + characters and device names. Implementers should research the + reserved names and characters for the types of storage device that + may be attached to their applications and restrict the use of data + obtained from URI components accordingly. + + + + + +Berners-Lee, et al. Standards Track [Page 44] + +RFC 3986 URI Generic Syntax January 2005 + + +7.4. Rare IP Address Formats + + Although the URI syntax for IPv4address only allows the common + dotted-decimal form of IPv4 address literal, many implementations + that process URIs make use of platform-dependent system routines, + such as gethostbyname() and inet_aton(), to translate the string + literal to an actual IP address. Unfortunately, such system routines + often allow and process a much larger set of formats than those + described in Section 3.2.2. + + For example, many implementations allow dotted forms of three + numbers, wherein the last part is interpreted as a 16-bit quantity + and placed in the right-most two bytes of the network address (e.g., + a Class B network). Likewise, a dotted form of two numbers means + that the last part is interpreted as a 24-bit quantity and placed in + the right-most three bytes of the network address (Class A), and a + single number (without dots) is interpreted as a 32-bit quantity and + stored directly in the network address. Adding further to the + confusion, some implementations allow each dotted part to be + interpreted as decimal, octal, or hexadecimal, as specified in the C + language (i.e., a leading 0x or 0X implies hexadecimal; a leading 0 + implies octal; otherwise, the number is interpreted as decimal). + + These additional IP address formats are not allowed in the URI syntax + due to differences between platform implementations. However, they + can become a security concern if an application attempts to filter + access to resources based on the IP address in string literal format. + If this filtering is performed, literals should be converted to + numeric form and filtered based on the numeric value, and not on a + prefix or suffix of the string form. + +7.5. Sensitive Information + + URI producers should not provide a URI that contains a username or + password that is intended to be secret. URIs are frequently + displayed by browsers, stored in clear text bookmarks, and logged by + user agent history and intermediary applications (proxies). A + password appearing within the userinfo component is deprecated and + should be considered an error (or simply ignored) except in those + rare cases where the 'password' parameter is intended to be public. + +7.6. Semantic Attacks + + Because the userinfo subcomponent is rarely used and appears before + the host in the authority component, it can be used to construct a + URI intended to mislead a human user by appearing to identify one + (trusted) naming authority while actually identifying a different + authority hidden behind the noise. For example + + + +Berners-Lee, et al. Standards Track [Page 45] + +RFC 3986 URI Generic Syntax January 2005 + + + ftp://cnn.example.com&story=breaking_news@10.0.0.1/top_story.htm + + might lead a human user to assume that the host is 'cnn.example.com', + whereas it is actually '10.0.0.1'. Note that a misleading userinfo + subcomponent could be much longer than the example above. + + A misleading URI, such as that above, is an attack on the user's + preconceived notions about the meaning of a URI rather than an attack + on the software itself. User agents may be able to reduce the impact + of such attacks by distinguishing the various components of the URI + when they are rendered, such as by using a different color or tone to + render userinfo if any is present, though there is no panacea. More + information on URI-based semantic attacks can be found in [Siedzik]. + +8. IANA Considerations + + URI scheme names, as defined by in Section 3.1, form a + registered namespace that is managed by IANA according to the + procedures defined in [BCP35]. No IANA actions are required by this + document. + +9. Acknowledgements + + This specification is derived from RFC 2396 [RFC2396], RFC 1808 + [RFC1808], and RFC 1738 [RFC1738]; the acknowledgements in those + documents still apply. It also incorporates the update (with + corrections) for IPv6 literals in the host syntax, as defined by + Robert M. Hinden, Brian E. Carpenter, and Larry Masinter in + [RFC2732]. In addition, contributions by Gisle Aas, Reese Anschultz, + Daniel Barclay, Tim Bray, Mike Brown, Rob Cameron, Jeremy Carroll, + Dan Connolly, Adam M. Costello, John Cowan, Jason Diamond, Martin + Duerst, Stefan Eissing, Clive D.W. Feather, Al Gilman, Tony Hammond, + Elliotte Harold, Pat Hayes, Henry Holtzman, Ian B. Jacobs, Michael + Kay, John C. Klensin, Graham Klyne, Dan Kohn, Bruce Lilly, Andrew + Main, Dave McAlpin, Ira McDonald, Michael Mealling, Ray Merkert, + Stephen Pollei, Julian Reschke, Tomas Rokicki, Miles Sabin, Kai + Schaetzl, Mark Thomson, Ronald Tschalaer, Norm Walsh, Marc Warne, + Stuart Williams, and Henry Zongaro are gratefully acknowledged. + +10. References + +10.1. Normative References + + [ASCII] American National Standards Institute, "Coded Character + Set -- 7-bit American Standard Code for Information + Interchange", ANSI X3.4, 1986. + + + + + +Berners-Lee, et al. Standards Track [Page 46] + +RFC 3986 URI Generic Syntax January 2005 + + + [RFC2234] Crocker, D. and P. Overell, "Augmented BNF for Syntax + Specifications: ABNF", RFC 2234, November 1997. + + [STD63] Yergeau, F., "UTF-8, a transformation format of + ISO 10646", STD 63, RFC 3629, November 2003. + + [UCS] International Organization for Standardization, + "Information Technology - Universal Multiple-Octet Coded + Character Set (UCS)", ISO/IEC 10646:2003, December 2003. + +10.2. Informative References + + [BCP19] Freed, N. and J. Postel, "IANA Charset Registration + Procedures", BCP 19, RFC 2978, October 2000. + + [BCP35] Petke, R. and I. King, "Registration Procedures for URL + Scheme Names", BCP 35, RFC 2717, November 1999. + + [RFC0952] Harrenstien, K., Stahl, M., and E. Feinler, "DoD Internet + host table specification", RFC 952, October 1985. + + [RFC1034] Mockapetris, P., "Domain names - concepts and facilities", + STD 13, RFC 1034, November 1987. + + [RFC1123] Braden, R., "Requirements for Internet Hosts - Application + and Support", STD 3, RFC 1123, October 1989. + + [RFC1535] Gavron, E., "A Security Problem and Proposed Correction + With Widely Deployed DNS Software", RFC 1535, + October 1993. + + [RFC1630] Berners-Lee, T., "Universal Resource Identifiers in WWW: A + Unifying Syntax for the Expression of Names and Addresses + of Objects on the Network as used in the World-Wide Web", + RFC 1630, June 1994. + + [RFC1736] Kunze, J., "Functional Recommendations for Internet + Resource Locators", RFC 1736, February 1995. + + [RFC1737] Sollins, K. and L. Masinter, "Functional Requirements for + Uniform Resource Names", RFC 1737, December 1994. + + [RFC1738] Berners-Lee, T., Masinter, L., and M. McCahill, "Uniform + Resource Locators (URL)", RFC 1738, December 1994. + + [RFC1808] Fielding, R., "Relative Uniform Resource Locators", + RFC 1808, June 1995. + + + + +Berners-Lee, et al. Standards Track [Page 47] + +RFC 3986 URI Generic Syntax January 2005 + + + [RFC2046] Freed, N. and N. Borenstein, "Multipurpose Internet Mail + Extensions (MIME) Part Two: Media Types", RFC 2046, + November 1996. + + [RFC2141] Moats, R., "URN Syntax", RFC 2141, May 1997. + + [RFC2396] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform + Resource Identifiers (URI): Generic Syntax", RFC 2396, + August 1998. + + [RFC2518] Goland, Y., Whitehead, E., Faizi, A., Carter, S., and D. + Jensen, "HTTP Extensions for Distributed Authoring -- + WEBDAV", RFC 2518, February 1999. + + [RFC2557] Palme, J., Hopmann, A., and N. Shelness, "MIME + Encapsulation of Aggregate Documents, such as HTML + (MHTML)", RFC 2557, March 1999. + + [RFC2718] Masinter, L., Alvestrand, H., Zigmond, D., and R. Petke, + "Guidelines for new URL Schemes", RFC 2718, November 1999. + + [RFC2732] Hinden, R., Carpenter, B., and L. Masinter, "Format for + Literal IPv6 Addresses in URL's", RFC 2732, December 1999. + + [RFC3305] Mealling, M. and R. Denenberg, "Report from the Joint + W3C/IETF URI Planning Interest Group: Uniform Resource + Identifiers (URIs), URLs, and Uniform Resource Names + (URNs): Clarifications and Recommendations", RFC 3305, + August 2002. + + [RFC3490] Faltstrom, P., Hoffman, P., and A. Costello, + "Internationalizing Domain Names in Applications (IDNA)", + RFC 3490, March 2003. + + [RFC3513] Hinden, R. and S. Deering, "Internet Protocol Version 6 + (IPv6) Addressing Architecture", RFC 3513, April 2003. + + [Siedzik] Siedzik, R., "Semantic Attacks: What's in a URL?", + April 2001, . + + + + + + + + + + + +Berners-Lee, et al. Standards Track [Page 48] + +RFC 3986 URI Generic Syntax January 2005 + + +Appendix A. Collected ABNF for URI + + URI = scheme ":" hier-part [ "?" query ] [ "#" fragment ] + + hier-part = "//" authority path-abempty + / path-absolute + / path-rootless + / path-empty + + URI-reference = URI / relative-ref + + absolute-URI = scheme ":" hier-part [ "?" query ] + + relative-ref = relative-part [ "?" query ] [ "#" fragment ] + + relative-part = "//" authority path-abempty + / path-absolute + / path-noscheme + / path-empty + + scheme = ALPHA *( ALPHA / DIGIT / "+" / "-" / "." ) + + authority = [ userinfo "@" ] host [ ":" port ] + userinfo = *( unreserved / pct-encoded / sub-delims / ":" ) + host = IP-literal / IPv4address / reg-name + port = *DIGIT + + IP-literal = "[" ( IPv6address / IPvFuture ) "]" + + IPvFuture = "v" 1*HEXDIG "." 1*( unreserved / sub-delims / ":" ) + + IPv6address = 6( h16 ":" ) ls32 + / "::" 5( h16 ":" ) ls32 + / [ h16 ] "::" 4( h16 ":" ) ls32 + / [ *1( h16 ":" ) h16 ] "::" 3( h16 ":" ) ls32 + / [ *2( h16 ":" ) h16 ] "::" 2( h16 ":" ) ls32 + / [ *3( h16 ":" ) h16 ] "::" h16 ":" ls32 + / [ *4( h16 ":" ) h16 ] "::" ls32 + / [ *5( h16 ":" ) h16 ] "::" h16 + / [ *6( h16 ":" ) h16 ] "::" + + h16 = 1*4HEXDIG + ls32 = ( h16 ":" h16 ) / IPv4address + IPv4address = dec-octet "." dec-octet "." dec-octet "." dec-octet + + + + + + + +Berners-Lee, et al. Standards Track [Page 49] + +RFC 3986 URI Generic Syntax January 2005 + + + dec-octet = DIGIT ; 0-9 + / %x31-39 DIGIT ; 10-99 + / "1" 2DIGIT ; 100-199 + / "2" %x30-34 DIGIT ; 200-249 + / "25" %x30-35 ; 250-255 + + reg-name = *( unreserved / pct-encoded / sub-delims ) + + path = path-abempty ; begins with "/" or is empty + / path-absolute ; begins with "/" but not "//" + / path-noscheme ; begins with a non-colon segment + / path-rootless ; begins with a segment + / path-empty ; zero characters + + path-abempty = *( "/" segment ) + path-absolute = "/" [ segment-nz *( "/" segment ) ] + path-noscheme = segment-nz-nc *( "/" segment ) + path-rootless = segment-nz *( "/" segment ) + path-empty = 0 + + segment = *pchar + segment-nz = 1*pchar + segment-nz-nc = 1*( unreserved / pct-encoded / sub-delims / "@" ) + ; non-zero-length segment without any colon ":" + + pchar = unreserved / pct-encoded / sub-delims / ":" / "@" + + query = *( pchar / "/" / "?" ) + + fragment = *( pchar / "/" / "?" ) + + pct-encoded = "%" HEXDIG HEXDIG + + unreserved = ALPHA / DIGIT / "-" / "." / "_" / "~" + reserved = gen-delims / sub-delims + gen-delims = ":" / "/" / "?" / "#" / "[" / "]" / "@" + sub-delims = "!" / "$" / "&" / "'" / "(" / ")" + / "*" / "+" / "," / ";" / "=" + +Appendix B. Parsing a URI Reference with a Regular Expression + + As the "first-match-wins" algorithm is identical to the "greedy" + disambiguation method used by POSIX regular expressions, it is + natural and commonplace to use a regular expression for parsing the + potential five components of a URI reference. + + The following line is the regular expression for breaking-down a + well-formed URI reference into its components. + + + +Berners-Lee, et al. Standards Track [Page 50] + +RFC 3986 URI Generic Syntax January 2005 + + + ^(([^:/?#]+):)?(//([^/?#]*))?([^?#]*)(\?([^#]*))?(#(.*))? + 12 3 4 5 6 7 8 9 + + The numbers in the second line above are only to assist readability; + they indicate the reference points for each subexpression (i.e., each + paired parenthesis). We refer to the value matched for subexpression + as $. For example, matching the above expression to + + http://www.ics.uci.edu/pub/ietf/uri/#Related + + results in the following subexpression matches: + + $1 = http: + $2 = http + $3 = //www.ics.uci.edu + $4 = www.ics.uci.edu + $5 = /pub/ietf/uri/ + $6 = + $7 = + $8 = #Related + $9 = Related + + where indicates that the component is not present, as is + the case for the query component in the above example. Therefore, we + can determine the value of the five components as + + scheme = $2 + authority = $4 + path = $5 + query = $7 + fragment = $9 + + Going in the opposite direction, we can recreate a URI reference from + its components by using the algorithm of Section 5.3. + +Appendix C. Delimiting a URI in Context + + URIs are often transmitted through formats that do not provide a + clear context for their interpretation. For example, there are many + occasions when a URI is included in plain text; examples include text + sent in email, USENET news, and on printed paper. In such cases, it + is important to be able to delimit the URI from the rest of the text, + and in particular from punctuation marks that might be mistaken for + part of the URI. + + In practice, URIs are delimited in a variety of ways, but usually + within double-quotes "http://example.com/", angle brackets + , or just by using whitespace: + + + +Berners-Lee, et al. Standards Track [Page 51] + +RFC 3986 URI Generic Syntax January 2005 + + + http://example.com/ + + These wrappers do not form part of the URI. + + In some cases, extra whitespace (spaces, line-breaks, tabs, etc.) may + have to be added to break a long URI across lines. The whitespace + should be ignored when the URI is extracted. + + No whitespace should be introduced after a hyphen ("-") character. + Because some typesetters and printers may (erroneously) introduce a + hyphen at the end of line when breaking it, the interpreter of a URI + containing a line break immediately after a hyphen should ignore all + whitespace around the line break and should be aware that the hyphen + may or may not actually be part of the URI. + + Using <> angle brackets around each URI is especially recommended as + a delimiting style for a reference that contains embedded whitespace. + + The prefix "URL:" (with or without a trailing space) was formerly + recommended as a way to help distinguish a URI from other bracketed + designators, though it is not commonly used in practice and is no + longer recommended. + + For robustness, software that accepts user-typed URI should attempt + to recognize and strip both delimiters and embedded whitespace. + + For example, the text + + Yes, Jim, I found it under "http://www.w3.org/Addressing/", + but you can probably pick it up from . Note the warning in . + + contains the URI references + + http://www.w3.org/Addressing/ + ftp://foo.example.com/rfc/ + http://www.ics.uci.edu/pub/ietf/uri/historical.html#WARNING + + + + + + + + + + + + + +Berners-Lee, et al. Standards Track [Page 52] + +RFC 3986 URI Generic Syntax January 2005 + + +Appendix D. Changes from RFC 2396 + +D.1. Additions + + An ABNF rule for URI has been introduced to correspond to one common + usage of the term: an absolute URI with optional fragment. + + IPv6 (and later) literals have been added to the list of possible + identifiers for the host portion of an authority component, as + described by [RFC2732], with the addition of "[" and "]" to the + reserved set and a version flag to anticipate future versions of IP + literals. Square brackets are now specified as reserved within the + authority component and are not allowed outside their use as + delimiters for an IP literal within host. In order to make this + change without changing the technical definition of the path, query, + and fragment components, those rules were redefined to directly + specify the characters allowed. + + As [RFC2732] defers to [RFC3513] for definition of an IPv6 literal + address, which, unfortunately, lacks an ABNF description of + IPv6address, we created a new ABNF rule for IPv6address that matches + the text representations defined by Section 2.2 of [RFC3513]. + Likewise, the definition of IPv4address has been improved in order to + limit each decimal octet to the range 0-255. + + Section 6, on URI normalization and comparison, has been completely + rewritten and extended by using input from Tim Bray and discussion + within the W3C Technical Architecture Group. + +D.2. Modifications + + The ad-hoc BNF syntax of RFC 2396 has been replaced with the ABNF of + [RFC2234]. This change required all rule names that formerly + included underscore characters to be renamed with a dash instead. In + addition, a number of syntax rules have been eliminated or simplified + to make the overall grammar more comprehensible. Specifications that + refer to the obsolete grammar rules may be understood by replacing + those rules according to the following table: + + + + + + + + + + + + + +Berners-Lee, et al. Standards Track [Page 53] + +RFC 3986 URI Generic Syntax January 2005 + + + +----------------+--------------------------------------------------+ + | obsolete rule | translation | + +----------------+--------------------------------------------------+ + | absoluteURI | absolute-URI | + | relativeURI | relative-part [ "?" query ] | + | hier_part | ( "//" authority path-abempty / | + | | path-absolute ) [ "?" query ] | + | | | + | opaque_part | path-rootless [ "?" query ] | + | net_path | "//" authority path-abempty | + | abs_path | path-absolute | + | rel_path | path-rootless | + | rel_segment | segment-nz-nc | + | reg_name | reg-name | + | server | authority | + | hostport | host [ ":" port ] | + | hostname | reg-name | + | path_segments | path-abempty | + | param | * | + | | | + | uric | unreserved / pct-encoded / ";" / "?" / ":" | + | | / "@" / "&" / "=" / "+" / "$" / "," / "/" | + | | | + | uric_no_slash | unreserved / pct-encoded / ";" / "?" / ":" | + | | / "@" / "&" / "=" / "+" / "$" / "," | + | | | + | mark | "-" / "_" / "." / "!" / "~" / "*" / "'" | + | | / "(" / ")" | + | | | + | escaped | pct-encoded | + | hex | HEXDIG | + | alphanum | ALPHA / DIGIT | + +----------------+--------------------------------------------------+ + + Use of the above obsolete rules for the definition of scheme-specific + syntax is deprecated. + + Section 2, on characters, has been rewritten to explain what + characters are reserved, when they are reserved, and why they are + reserved, even when they are not used as delimiters by the generic + syntax. The mark characters that are typically unsafe to decode, + including the exclamation mark ("!"), asterisk ("*"), single-quote + ("'"), and open and close parentheses ("(" and ")"), have been moved + to the reserved set in order to clarify the distinction between + reserved and unreserved and, hopefully, to answer the most common + question of scheme designers. Likewise, the section on + percent-encoded characters has been rewritten, and URI normalizers + are now given license to decode any percent-encoded octets + + + +Berners-Lee, et al. Standards Track [Page 54] + +RFC 3986 URI Generic Syntax January 2005 + + + corresponding to unreserved characters. In general, the terms + "escaped" and "unescaped" have been replaced with "percent-encoded" + and "decoded", respectively, to reduce confusion with other forms of + escape mechanisms. + + The ABNF for URI and URI-reference has been redesigned to make them + more friendly to LALR parsers and to reduce complexity. As a result, + the layout form of syntax description has been removed, along with + the uric, uric_no_slash, opaque_part, net_path, abs_path, rel_path, + path_segments, rel_segment, and mark rules. All references to + "opaque" URIs have been replaced with a better description of how the + path component may be opaque to hierarchy. The relativeURI rule has + been replaced with relative-ref to avoid unnecessary confusion over + whether they are a subset of URI. The ambiguity regarding the + parsing of URI-reference as a URI or a relative-ref with a colon in + the first segment has been eliminated through the use of five + separate path matching rules. + + The fragment identifier has been moved back into the section on + generic syntax components and within the URI and relative-ref rules, + though it remains excluded from absolute-URI. The number sign ("#") + character has been moved back to the reserved set as a result of + reintegrating the fragment syntax. + + The ABNF has been corrected to allow the path component to be empty. + This also allows an absolute-URI to consist of nothing after the + "scheme:", as is present in practice with the "dav:" namespace + [RFC2518] and with the "about:" scheme used internally by many WWW + browser implementations. The ambiguity regarding the boundary + between authority and path has been eliminated through the use of + five separate path matching rules. + + Registry-based naming authorities that use the generic syntax are now + defined within the host rule. This change allows current + implementations, where whatever name provided is simply fed to the + local name resolution mechanism, to be consistent with the + specification. It also removes the need to re-specify DNS name + formats here. Furthermore, it allows the host component to contain + percent-encoded octets, which is necessary to enable + internationalized domain names to be provided in URIs, processed in + their native character encodings at the application layers above URI + processing, and passed to an IDNA library as a registered name in the + UTF-8 character encoding. The server, hostport, hostname, + domainlabel, toplabel, and alphanum rules have been removed. + + The resolving relative references algorithm of [RFC2396] has been + rewritten with pseudocode for this revision to improve clarity and + fix the following issues: + + + +Berners-Lee, et al. Standards Track [Page 55] + +RFC 3986 URI Generic Syntax January 2005 + + + o [RFC2396] section 5.2, step 6a, failed to account for a base URI + with no path. + + o Restored the behavior of [RFC1808] where, if the reference + contains an empty path and a defined query component, the target + URI inherits the base URI's path component. + + o The determination of whether a URI reference is a same-document + reference has been decoupled from the URI parser, simplifying the + URI processing interface within applications in a way consistent + with the internal architecture of deployed URI processing + implementations. The determination is now based on comparison to + the base URI after transforming a reference to absolute form, + rather than on the format of the reference itself. This change + may result in more references being considered "same-document" + under this specification than there would be under the rules given + in RFC 2396, especially when normalization is used to reduce + aliases. However, it does not change the status of existing + same-document references. + + o Separated the path merge routine into two routines: merge, for + describing combination of the base URI path with a relative-path + reference, and remove_dot_segments, for describing how to remove + the special "." and ".." segments from a composed path. The + remove_dot_segments algorithm is now applied to all URI reference + paths in order to match common implementations and to improve the + normalization of URIs in practice. This change only impacts the + parsing of abnormal references and same-scheme references wherein + the base URI has a non-hierarchical path. + +Index + + A + ABNF 11 + absolute 27 + absolute-path 26 + absolute-URI 27 + access 9 + authority 17, 18 + + B + base URI 28 + + C + character encoding 4 + character 4 + characters 8, 11 + coded character set 4 + + + +Berners-Lee, et al. Standards Track [Page 56] + +RFC 3986 URI Generic Syntax January 2005 + + + D + dec-octet 20 + dereference 9 + dot-segments 23 + + F + fragment 16, 24 + + G + gen-delims 13 + generic syntax 6 + + H + h16 20 + hier-part 16 + hierarchical 10 + host 18 + + I + identifier 5 + IP-literal 19 + IPv4 20 + IPv4address 19, 20 + IPv6 19 + IPv6address 19, 20 + IPvFuture 19 + + L + locator 7 + ls32 20 + + M + merge 32 + + N + name 7 + network-path 26 + + P + path 16, 22, 26 + path-abempty 22 + path-absolute 22 + path-empty 22 + path-noscheme 22 + path-rootless 22 + path-abempty 16, 22, 26 + path-absolute 16, 22, 26 + path-empty 16, 22, 26 + + + +Berners-Lee, et al. Standards Track [Page 57] + +RFC 3986 URI Generic Syntax January 2005 + + + path-rootless 16, 22 + pchar 23 + pct-encoded 12 + percent-encoding 12 + port 22 + + Q + query 16, 23 + + R + reg-name 21 + registered name 20 + relative 10, 28 + relative-path 26 + relative-ref 26 + remove_dot_segments 33 + representation 9 + reserved 12 + resolution 9, 28 + resource 5 + retrieval 9 + + S + same-document 27 + sameness 9 + scheme 16, 17 + segment 22, 23 + segment-nz 23 + segment-nz-nc 23 + sub-delims 13 + suffix 27 + + T + transcription 8 + + U + uniform 4 + unreserved 13 + URI grammar + absolute-URI 27 + ALPHA 11 + authority 18 + CR 11 + dec-octet 20 + DIGIT 11 + DQUOTE 11 + fragment 24 + gen-delims 13 + + + +Berners-Lee, et al. Standards Track [Page 58] + +RFC 3986 URI Generic Syntax January 2005 + + + h16 20 + HEXDIG 11 + hier-part 16 + host 19 + IP-literal 19 + IPv4address 20 + IPv6address 20 + IPvFuture 19 + LF 11 + ls32 20 + OCTET 11 + path 22 + path-abempty 22 + path-absolute 22 + path-empty 22 + path-noscheme 22 + path-rootless 22 + pchar 23 + pct-encoded 12 + port 22 + query 24 + reg-name 21 + relative-ref 26 + reserved 13 + scheme 17 + segment 23 + segment-nz 23 + segment-nz-nc 23 + SP 11 + sub-delims 13 + unreserved 13 + URI 16 + URI-reference 25 + userinfo 18 + URI 16 + URI-reference 25 + URL 7 + URN 7 + userinfo 18 + + + + + + + + + + + + +Berners-Lee, et al. Standards Track [Page 59] + +RFC 3986 URI Generic Syntax January 2005 + + +Authors' Addresses + + Tim Berners-Lee + World Wide Web Consortium + Massachusetts Institute of Technology + 77 Massachusetts Avenue + Cambridge, MA 02139 + USA + + Phone: +1-617-253-5702 + Fax: +1-617-258-5999 + EMail: timbl@w3.org + URI: http://www.w3.org/People/Berners-Lee/ + + + Roy T. Fielding + Day Software + 5251 California Ave., Suite 110 + Irvine, CA 92617 + USA + + Phone: +1-949-679-2960 + Fax: +1-949-679-2972 + EMail: fielding@gbiv.com + URI: http://roy.gbiv.com/ + + + Larry Masinter + Adobe Systems Incorporated + 345 Park Ave + San Jose, CA 95110 + USA + + Phone: +1-408-536-3024 + EMail: LMM@acm.org + URI: http://larry.masinter.net/ + + + + + + + + + + + + + + + +Berners-Lee, et al. Standards Track [Page 60] + +RFC 3986 URI Generic Syntax January 2005 + + +Full Copyright Statement + + Copyright (C) The Internet Society (2005). + + This document is subject to the rights, licenses and restrictions + contained in BCP 78, and except as set forth therein, the authors + retain all their rights. + + This document and the information contained herein are provided on an + "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS + OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET + ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, + INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE + INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED + WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. + +Intellectual Property + + The IETF takes no position regarding the validity or scope of any + Intellectual Property Rights or other rights that might be claimed to + pertain to the implementation or use of the technology described in + this document or the extent to which any license under such rights + might or might not be available; nor does it represent that it has + made any independent effort to identify any such rights. Information + on the IETF's procedures with respect to rights in IETF Documents can + be found in BCP 78 and BCP 79. + + Copies of IPR disclosures made to the IETF Secretariat and any + assurances of licenses to be made available, or the result of an + attempt made to obtain a general license or permission for the use of + such proprietary rights by implementers or users of this + specification can be obtained from the IETF on-line IPR repository at + http://www.ietf.org/ipr. + + The IETF invites any interested party to bring to its attention any + copyrights, patents or patent applications, or other proprietary + rights that may cover technology that may be required to implement + this standard. Please address the information to the IETF at ietf- + ipr@ietf.org. + + +Acknowledgement + + Funding for the RFC Editor function is currently provided by the + Internet Society. + + + + + + +Berners-Lee, et al. Standards Track [Page 61] + diff --git a/docs/rfc3987.txt b/docs/rfc3987.txt new file mode 100644 index 0000000..f0b1513 --- /dev/null +++ b/docs/rfc3987.txt @@ -0,0 +1,2579 @@ + + + + + + +Network Working Group M. Duerst +Request for Comments: 3987 W3C +Category: Standards Track M. Suignard + Microsoft Corporation + January 2005 + + + Internationalized Resource Identifiers (IRIs) + +Status of This Memo + + This document specifies an Internet standards track protocol for the + Internet community, and requests discussion and suggestions for + improvements. Please refer to the current edition of the "Internet + Official Protocol Standards" (STD 1) for the standardization state + and status of this protocol. Distribution of this memo is unlimited. + +Copyright Notice + + Copyright (C) The Internet Society (2005). + +Abstract + + This document defines a new protocol element, the Internationalized + Resource Identifier (IRI), as a complement to the Uniform Resource + Identifier (URI). An IRI is a sequence of characters from the + Universal Character Set (Unicode/ISO 10646). A mapping from IRIs to + URIs is defined, which means that IRIs can be used instead of URIs, + where appropriate, to identify resources. + + The approach of defining a new protocol element was chosen instead of + extending or changing the definition of URIs. This was done in order + to allow a clear distinction and to avoid incompatibilities with + existing software. Guidelines are provided for the use and + deployment of IRIs in various protocols, formats, and software + components that currently deal with URIs. + +Table of Contents + + 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 + 1.1. Overview and Motivation . . . . . . . . . . . . . . . . 3 + 1.2. Applicability . . . . . . . . . . . . . . . . . . . . . 3 + 1.3. Definitions . . . . . . . . . . . . . . . . . . . . . . 4 + 1.4. Notation . . . . . . . . . . . . . . . . . . . . . . . . 5 + 2. IRI Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . 6 + 2.1. Summary of IRI Syntax . . . . . . . . . . . . . . . . . 6 + 2.2. ABNF for IRI References and IRIs . . . . . . . . . . . . 7 + + + + +Duerst & Suignard Standards Track [Page 1] + +RFC 3987 Internationalized Resource Identifiers January 2005 + + + 3. Relationship between IRIs and URIs . . . . . . . . . . . . . . 10 + 3.1. Mapping of IRIs to URIs . . . . . . . . . . . . . . . . 10 + 3.2. Converting URIs to IRIs . . . . . . . . . . . . . . . . 14 + 3.2.1. Examples . . . . . . . . . . . . . . . . . . . . 15 + 4. Bidirectional IRIs for Right-to-Left Languages. . . . . . . . 16 + 4.1. Logical Storage and Visual Presentation . . . . . . . . 17 + 4.2. Bidi IRI Structure . . . . . . . . . . . . . . . . . . . 18 + 4.3. Input of Bidi IRIs . . . . . . . . . . . . . . . . . . . 19 + 4.4. Examples . . . . . . . . . . . . . . . . . . . . . . . . 19 + 5. Normalization and Comparison . . . . . . . . . . . . . . . . . 21 + 5.1. Equivalence . . . . . . . . . . . . . . . . . . . . . . 22 + 5.2. Preparation for Comparison . . . . . . . . . . . . . . . 22 + 5.3. Comparison Ladder . . . . . . . . . . . . . . . . . . . 23 + 5.3.1. Simple String Comparison . . . . . . . . . . . . 23 + 5.3.2. Syntax-Based Normalization . . . . . . . . . . . 24 + 5.3.3. Scheme-Based Normalization . . . . . . . . . . . 27 + 5.3.4. Protocol-Based Normalization . . . . . . . . . . 28 + 6. Use of IRIs . . . . . . . . . . . . . . . . . . . . . . . . . 29 + 6.1. Limitations on UCS Characters Allowed in IRIs . . . . . 29 + 6.2. Software Interfaces and Protocols . . . . . . . . . . . 29 + 6.3. Format of URIs and IRIs in Documents and Protocols . . . 30 + 6.4. Use of UTF-8 for Encoding Original Characters .. . . . . 30 + 6.5. Relative IRI References . . . . . . . . . . . . . . . . 32 + 7. URI/IRI Processing Guidelines (informative) . . . . . . . . . 32 + 7.1. URI/IRI Software Interfaces . . . . . . . . . . . . . . 32 + 7.2. URI/IRI Entry . . . . . . . . . . . . . . . . . . . . . 33 + 7.3. URI/IRI Transfer between Applications . . . . . . . . . 33 + 7.4. URI/IRI Generation . . . . . . . . . . . . . . . . . . . 34 + 7.5. URI/IRI Selection . . . . . . . . . . . . . . . . . . . 34 + 7.6. Display of URIs/IRIs . . . . . . . . . . . . . . . . . . 35 + 7.7. Interpretation of URIs and IRIs . . . . . . . . . . . . 36 + 7.8. Upgrading Strategy . . . . . . . . . . . . . . . . . . . 36 + 8. Security Considerations . . . . . . . . . . . . . . . . . . . 37 + 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 39 + 10. References . . . . . . . . . . . . . . . . . . . . . . . . . . 40 + 10.1. Normative References . . . . . . . . . . . . . . . . . . 40 + 10.2. Informative References . . . . . . . . . . . . . . . . . 41 + A. Design Alternatives . . . . . . . . . . . . . . . . . . . . . 44 + A.1. New Scheme(s) . . . . . . . . . . . . . . . . . . . . . 44 + A.2. Character Encodings Other Than UTF-8 . . . . . . . . . . 44 + A.3. New Encoding Convention . . . . . . . . . . . . . . . . 44 + A.4. Indicating Character Encodings in the URI/IRI . . . . . 45 + Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 45 + Full Copyright Statement . . . . . . . . . . . . . . . . . . . . . 46 + + + + + + + +Duerst & Suignard Standards Track [Page 2] + +RFC 3987 Internationalized Resource Identifiers January 2005 + + +1. Introduction + +1.1. Overview and Motivation + + A Uniform Resource Identifier (URI) is defined in [RFC3986] as a + sequence of characters chosen from a limited subset of the repertoire + of US-ASCII [ASCII] characters. + + The characters in URIs are frequently used for representing words of + natural languages. This usage has many advantages: Such URIs are + easier to memorize, easier to interpret, easier to transcribe, easier + to create, and easier to guess. For most languages other than + English, however, the natural script uses characters other than A - + Z. For many people, handling Latin characters is as difficult as + handling the characters of other scripts is for those who use only + the Latin alphabet. Many languages with non-Latin scripts are + transcribed with Latin letters. These transcriptions are now often + used in URIs, but they introduce additional ambiguities. + + The infrastructure for the appropriate handling of characters from + local scripts is now widely deployed in local versions of operating + system and application software. Software that can handle a wide + variety of scripts and languages at the same time is increasingly + common. Also, increasing numbers of protocols and formats can carry + a wide range of characters. + + This document defines a new protocol element called Internationalized + Resource Identifier (IRI) by extending the syntax of URIs to a much + wider repertoire of characters. It also defines "internationalized" + versions corresponding to other constructs from [RFC3986], such as + URI references. The syntax of IRIs is defined in section 2, and the + relationship between IRIs and URIs in section 3. + + Using characters outside of A - Z in IRIs brings some difficulties. + Section 4 discusses the special case of bidirectional IRIs, section 5 + various forms of equivalence between IRIs, and section 6 the use of + IRIs in different situations. Section 7 gives additional informative + guidelines, and section 8 security considerations. + +1.2. Applicability + + IRIs are designed to be compatible with recommendations for new URI + schemes [RFC2718]. The compatibility is provided by specifying a + well-defined and deterministic mapping from the IRI character + sequence to the functionally equivalent URI character sequence. + Practical use of IRIs (or IRI references) in place of URIs (or URI + references) depends on the following conditions being met: + + + + +Duerst & Suignard Standards Track [Page 3] + +RFC 3987 Internationalized Resource Identifiers January 2005 + + + a. A protocol or format element should be explicitly designated to + be able to carry IRIs. The intent is not to introduce IRIs into + contexts that are not defined to accept them. For example, XML + schema [XMLSchema] has an explicit type "anyURI" that includes + IRIs and IRI references. Therefore, IRIs and IRI references can + be in attributes and elements of type "anyURI". On the other + hand, in the HTTP protocol [RFC2616], the Request URI is defined + as a URI, which means that direct use of IRIs is not allowed in + HTTP requests. + + b. The protocol or format carrying the IRIs should have a mechanism + to represent the wide range of characters used in IRIs, either + natively or by some protocol- or format-specific escaping + mechanism (for example, numeric character references in [XML1]). + + c. The URI corresponding to the IRI in question has to encode + original characters into octets using UTF-8. For new URI + schemes, this is recommended in [RFC2718]. It can apply to a + whole scheme (e.g., IMAP URLs [RFC2192] and POP URLs [RFC2384], + or the URN syntax [RFC2141]). It can apply to a specific part of + a URI, such as the fragment identifier (e.g., [XPointer]). It + can apply to a specific URI or part(s) thereof. For details, + please see section 6.4. + +1.3. Definitions + + The following definitions are used in this document; they follow the + terms in [RFC2130], [RFC2277], and [ISO10646]. + + character: A member of a set of elements used for the organization, + control, or representation of data. For example, "LATIN CAPITAL + LETTER A" names a character. + + octet: An ordered sequence of eight bits considered as a unit. + + character repertoire: A set of characters (in the mathematical + sense). + + sequence of characters: A sequence of characters (one after another). + + sequence of octets: A sequence of octets (one after another). + + character encoding: A method of representing a sequence of characters + as a sequence of octets (maybe with variants). Also, a method of + (unambiguously) converting a sequence of octets into a sequence of + characters. + + + + + +Duerst & Suignard Standards Track [Page 4] + +RFC 3987 Internationalized Resource Identifiers January 2005 + + + charset: The name of a parameter or attribute used to identify a + character encoding. + + UCS: Universal Character Set. The coded character set defined by + ISO/IEC 10646 [ISO10646] and the Unicode Standard [UNIV4]. + + IRI reference: Denotes the common usage of an Internationalized + Resource Identifier. An IRI reference may be absolute or + relative. However, the "IRI" that results from such a reference + only includes absolute IRIs; any relative IRI references are + resolved to their absolute form. Note that in [RFC2396] URIs did + not include fragment identifiers, but in [RFC3986] fragment + identifiers are part of URIs. + + running text: Human text (paragraphs, sentences, phrases) with syntax + according to orthographic conventions of a natural language, as + opposed to syntax defined for ease of processing by machines + (e.g., markup, programming languages). + + protocol element: Any portion of a message that affects processing of + that message by the protocol in question. + + presentation element: A presentation form corresponding to a protocol + element; for example, using a wider range of characters. + + create (a URI or IRI): With respect to URIs and IRIs, the term is + used for the initial creation. This may be the initial creation + of a resource with a certain identifier, or the initial exposition + of a resource under a particular identifier. + + generate (a URI or IRI): With respect to URIs and IRIs, the term is + used when the IRI is generated by derivation from other + information. + +1.4. Notation + + RFCs and Internet Drafts currently do not allow any characters + outside the US-ASCII repertoire. Therefore, this document uses + various special notations to denote such characters in examples. + + In text, characters outside US-ASCII are sometimes referenced by + using a prefix of 'U+', followed by four to six hexadecimal digits. + + To represent characters outside US-ASCII in examples, this document + uses two notations: 'XML Notation' and 'Bidi Notation'. + + + + + + +Duerst & Suignard Standards Track [Page 5] + +RFC 3987 Internationalized Resource Identifiers January 2005 + + + XML Notation uses a leading '&#x', a trailing ';', and the + hexadecimal number of the character in the UCS in between. For + example, я stands for CYRILLIC CAPITAL LETTER YA. In this + notation, an actual '&' is denoted by '&'. + + Bidi Notation is used for bidirectional examples: Lowercase letters + stand for Latin letters or other letters that are written left to + right, whereas uppercase letters represent Arabic or Hebrew letters + that are written right to left. + + To denote actual octets in examples (as opposed to percent-encoded + octets), the two hex digits denoting the octet are enclosed in "<" + and ">". For example, the octet often denoted as 0xc9 is denoted + here as . + + In this document, the key words "MUST", "MUST NOT", "REQUIRED", + "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", + and "OPTIONAL" are to be interpreted as described in [RFC2119]. + +2. IRI Syntax + + This section defines the syntax of Internationalized Resource + Identifiers (IRIs). + + As with URIs, an IRI is defined as a sequence of characters, not as a + sequence of octets. This definition accommodates the fact that IRIs + may be written on paper or read over the radio as well as stored or + transmitted digitally. The same IRI may be represented as different + sequences of octets in different protocols or documents if these + protocols or documents use different character encodings (and/or + transfer encodings). Using the same character encoding as the + containing protocol or document ensures that the characters in the + IRI can be handled (e.g., searched, converted, displayed) in the same + way as the rest of the protocol or document. + +2.1. Summary of IRI Syntax + + IRIs are defined similarly to URIs in [RFC3986], but the class of + unreserved characters is extended by adding the characters of the UCS + (Universal Character Set, [ISO10646]) beyond U+007F, subject to the + limitations given in the syntax rules below and in section 6.1. + + Otherwise, the syntax and use of components and reserved characters + is the same as that in [RFC3986]. All the operations defined in + [RFC3986], such as the resolution of relative references, can be + applied to IRIs by IRI-processing software in exactly the same way as + they are for URIs by URI-processing software. + + + + +Duerst & Suignard Standards Track [Page 6] + +RFC 3987 Internationalized Resource Identifiers January 2005 + + + Characters outside the US-ASCII repertoire are not reserved and + therefore MUST NOT be used for syntactical purposes, such as to + delimit components in newly defined schemes. For example, U+00A2, + CENT SIGN, is not allowed as a delimiter in IRIs, because it is in + the 'iunreserved' category. This is similar to the fact that it is + not possible to use '-' as a delimiter in URIs, because it is in the + 'unreserved' category. + +2.2. ABNF for IRI References and IRIs + + Although it might be possible to define IRI references and IRIs + merely by their transformation to URI references and URIs, they can + also be accepted and processed directly. Therefore, an ABNF + definition for IRI references (which are the most general concept and + the start of the grammar) and IRIs is given here. The syntax of this + ABNF is described in [RFC2234]. Character numbers are taken from the + UCS, without implying any actual binary encoding. Terminals in the + ABNF are characters, not bytes. + + The following grammar closely follows the URI grammar in [RFC3986], + except that the range of unreserved characters is expanded to include + UCS characters, with the restriction that private UCS characters can + occur only in query parts. The grammar is split into two parts: + Rules that differ from [RFC3986] because of the above-mentioned + expansion, and rules that are the same as those in [RFC3986]. For + rules that are different than those in [RFC3986], the names of the + non-terminals have been changed as follows. If the non-terminal + contains 'URI', this has been changed to 'IRI'. Otherwise, an 'i' + has been prefixed. + + The following rules are different from those in [RFC3986]: + + IRI = scheme ":" ihier-part [ "?" iquery ] + [ "#" ifragment ] + + ihier-part = "//" iauthority ipath-abempty + / ipath-absolute + / ipath-rootless + / ipath-empty + + IRI-reference = IRI / irelative-ref + + absolute-IRI = scheme ":" ihier-part [ "?" iquery ] + + irelative-ref = irelative-part [ "?" iquery ] [ "#" ifragment ] + + irelative-part = "//" iauthority ipath-abempty + / ipath-absolute + + + +Duerst & Suignard Standards Track [Page 7] + +RFC 3987 Internationalized Resource Identifiers January 2005 + + + / ipath-noscheme + / ipath-empty + + iauthority = [ iuserinfo "@" ] ihost [ ":" port ] + iuserinfo = *( iunreserved / pct-encoded / sub-delims / ":" ) + ihost = IP-literal / IPv4address / ireg-name + + ireg-name = *( iunreserved / pct-encoded / sub-delims ) + + ipath = ipath-abempty ; begins with "/" or is empty + / ipath-absolute ; begins with "/" but not "//" + / ipath-noscheme ; begins with a non-colon segment + / ipath-rootless ; begins with a segment + / ipath-empty ; zero characters + + ipath-abempty = *( "/" isegment ) + ipath-absolute = "/" [ isegment-nz *( "/" isegment ) ] + ipath-noscheme = isegment-nz-nc *( "/" isegment ) + ipath-rootless = isegment-nz *( "/" isegment ) + ipath-empty = 0 + + isegment = *ipchar + isegment-nz = 1*ipchar + isegment-nz-nc = 1*( iunreserved / pct-encoded / sub-delims + / "@" ) + ; non-zero-length segment without any colon ":" + + ipchar = iunreserved / pct-encoded / sub-delims / ":" + / "@" + + iquery = *( ipchar / iprivate / "/" / "?" ) + + ifragment = *( ipchar / "/" / "?" ) + + iunreserved = ALPHA / DIGIT / "-" / "." / "_" / "~" / ucschar + + ucschar = %xA0-D7FF / %xF900-FDCF / %xFDF0-FFEF + / %x10000-1FFFD / %x20000-2FFFD / %x30000-3FFFD + / %x40000-4FFFD / %x50000-5FFFD / %x60000-6FFFD + / %x70000-7FFFD / %x80000-8FFFD / %x90000-9FFFD + / %xA0000-AFFFD / %xB0000-BFFFD / %xC0000-CFFFD + / %xD0000-DFFFD / %xE1000-EFFFD + + iprivate = %xE000-F8FF / %xF0000-FFFFD / %x100000-10FFFD + + Some productions are ambiguous. The "first-match-wins" (a.k.a. + "greedy") algorithm applies. For details, see [RFC3986]. + + + + +Duerst & Suignard Standards Track [Page 8] + +RFC 3987 Internationalized Resource Identifiers January 2005 + + + The following rules are the same as those in [RFC3986]: + + scheme = ALPHA *( ALPHA / DIGIT / "+" / "-" / "." ) + + port = *DIGIT + + IP-literal = "[" ( IPv6address / IPvFuture ) "]" + + IPvFuture = "v" 1*HEXDIG "." 1*( unreserved / sub-delims / ":" ) + + IPv6address = 6( h16 ":" ) ls32 + / "::" 5( h16 ":" ) ls32 + / [ h16 ] "::" 4( h16 ":" ) ls32 + / [ *1( h16 ":" ) h16 ] "::" 3( h16 ":" ) ls32 + / [ *2( h16 ":" ) h16 ] "::" 2( h16 ":" ) ls32 + / [ *3( h16 ":" ) h16 ] "::" h16 ":" ls32 + / [ *4( h16 ":" ) h16 ] "::" ls32 + / [ *5( h16 ":" ) h16 ] "::" h16 + / [ *6( h16 ":" ) h16 ] "::" + + h16 = 1*4HEXDIG + ls32 = ( h16 ":" h16 ) / IPv4address + + IPv4address = dec-octet "." dec-octet "." dec-octet "." dec-octet + + dec-octet = DIGIT ; 0-9 + / %x31-39 DIGIT ; 10-99 + / "1" 2DIGIT ; 100-199 + / "2" %x30-34 DIGIT ; 200-249 + / "25" %x30-35 ; 250-255 + + pct-encoded = "%" HEXDIG HEXDIG + + unreserved = ALPHA / DIGIT / "-" / "." / "_" / "~" + reserved = gen-delims / sub-delims + gen-delims = ":" / "/" / "?" / "#" / "[" / "]" / "@" + sub-delims = "!" / "$" / "&" / "'" / "(" / ")" + / "*" / "+" / "," / ";" / "=" + + This syntax does not support IPv6 scoped addressing zone identifiers. + + + + + + + + + + + +Duerst & Suignard Standards Track [Page 9] + +RFC 3987 Internationalized Resource Identifiers January 2005 + + +3. Relationship between IRIs and URIs + + IRIs are meant to replace URIs in identifying resources for + protocols, formats, and software components that use a UCS-based + character repertoire. These protocols and components may never need + to use URIs directly, especially when the resource identifier is used + simply for identification purposes. However, when the resource + identifier is used for resource retrieval, it is in many cases + necessary to determine the associated URI, because currently most + retrieval mechanisms are only defined for URIs. In this case, IRIs + can serve as presentation elements for URI protocol elements. An + example would be an address bar in a Web user agent. (Additional + rationale is given in section 3.1.) + +3.1. Mapping of IRIs to URIs + + This section defines how to map an IRI to a URI. Everything in this + section also applies to IRI references and URI references, as well as + to components thereof (for example, fragment identifiers). + + This mapping has two purposes: + + Syntaxical. Many URI schemes and components define additional + syntactical restrictions not captured in section 2.2. + Scheme-specific restrictions are applied to IRIs by converting + IRIs to URIs and checking the URIs against the scheme-specific + restrictions. + + Interpretational. URIs identify resources in various ways. IRIs also + identify resources. When the IRI is used solely for + identification purposes, it is not necessary to map the IRI to a + URI (see section 5). However, when an IRI is used for resource + retrieval, the resource that the IRI locates is the same as the + one located by the URI obtained after converting the IRI according + to the procedure defined here. This means that there is no need + to define resolution separately on the IRI level. + + Applications MUST map IRIs to URIs by using the following two steps. + + Step 1. Generate a UCS character sequence from the original IRI + format. This step has the following three variants, + depending on the form of the input: + + a. If the IRI is written on paper, read aloud, or otherwise + represented as a sequence of characters independent of + any character encoding, represent the IRI as a sequence + of characters from the UCS normalized according to + Normalization Form C (NFC, [UTR15]). + + + +Duerst & Suignard Standards Track [Page 10] + +RFC 3987 Internationalized Resource Identifiers January 2005 + + + b. If the IRI is in some digital representation (e.g., an + octet stream) in some known non-Unicode character + encoding, convert the IRI to a sequence of characters + from the UCS normalized according to NFC. + + c. If the IRI is in a Unicode-based character encoding (for + example, UTF-8 or UTF-16), do not normalize (see section + 5.3.2.2 for details). Apply step 2 directly to the + encoded Unicode character sequence. + + Step 2. For each character in 'ucschar' or 'iprivate', apply steps + 2.1 through 2.3 below. + + 2.1. Convert the character to a sequence of one or more octets + using UTF-8 [RFC3629]. + + 2.2. Convert each octet to %HH, where HH is the hexadecimal + notation of the octet value. Note that this is identical + to the percent-encoding mechanism in section 2.1 of + [RFC3986]. To reduce variability, the hexadecimal notation + SHOULD use uppercase letters. + + 2.3. Replace the original character with the resulting character + sequence (i.e., a sequence of %HH triplets). + + The above mapping from IRIs to URIs produces URIs fully conforming to + [RFC3986]. The mapping is also an identity transformation for URIs + and is idempotent; applying the mapping a second time will not + change anything. Every URI is by definition an IRI. + + Systems accepting IRIs MAY convert the ireg-name component of an IRI + as follows (before step 2 above) for schemes known to use domain + names in ireg-name, if the scheme definition does not allow + percent-encoding for ireg-name: + + Replace the ireg-name part of the IRI by the part converted using the + ToASCII operation specified in section 4.1 of [RFC3490] on each + dot-separated label, and by using U+002E (FULL STOP) as a label + separator, with the flag UseSTD3ASCIIRules set to TRUE, and with the + flag AllowUnassigned set to FALSE for creating IRIs and set to TRUE + otherwise. + + + + + + + + + + +Duerst & Suignard Standards Track [Page 11] + +RFC 3987 Internationalized Resource Identifiers January 2005 + + + The ToASCII operation may fail, but this would mean that the IRI + cannot be resolved. This conversion SHOULD be used when the goal is + to maximize interoperability with legacy URI resolvers. For example, + the IRI + + "http://résumé.example.org" + + may be converted to + + "http://xn--rsum-bpad.example.org" + + instead of + + "http://r%C3%A9sum%C3%A9.example.org". + + An IRI with a scheme that is known to use domain names in ireg-name, + but where the scheme definition does not allow percent-encoding for + ireg-name, meets scheme-specific restrictions if either the + straightforward conversion or the conversion using the ToASCII + operation on ireg-name result in an URI that meets the scheme- + specific restrictions. + + Such an IRI resolves to the URI obtained after converting the IRI and + uses the ToASCII operation on ireg-name. Implementations do not have + to do this conversion as long as they produce the same result. + + Note: The difference between variants b and c in step 1 (using + normalization with NFC, versus not using any normalization) + accounts for the fact that in many non-Unicode character + encodings, some text cannot be represented directly. For example, + the word "Vietnam" is natively written "Việt Nam" + (containing a LATIN SMALL LETTER E WITH CIRCUMFLEX AND DOT BELOW) + in NFC, but a direct transcoding from the windows-1258 character + encoding leads to "Việt Nam" (containing a LATIN SMALL + LETTER E WITH CIRCUMFLEX followed by a COMBINING DOT BELOW). + Direct transcoding of other 8-bit encodings of Vietnamese may lead + to other representations. + + Note: The uniform treatment of the whole IRI in step 2 is important + to make processing independent of URI scheme. See [Gettys] for an + in-depth discussion. + + Note: In practice, whether the general mapping (steps 1 and 2) or the + ToASCII operation of [RFC3490] is used for ireg-name will not be + noticed if mapping from IRI to URI and resolution is tightly + integrated (e.g., carried out in the same user agent). But + + + + + +Duerst & Suignard Standards Track [Page 12] + +RFC 3987 Internationalized Resource Identifiers January 2005 + + + conversion using [RFC3490] may be able to better deal with + backwards compatibility issues in case mapping and resolution are + separated, as in the case of using an HTTP proxy. + + Note: Internationalized Domain Names may be contained in parts of an + IRI other than the ireg-name part. It is the responsibility of + scheme-specific implementations (if the Internationalized Domain + Name is part of the scheme syntax) or of server-side + implementations (if the Internationalized Domain Name is part of + 'iquery') to apply the necessary conversions at the appropriate + point. Example: Trying to validate the Web page at + http://résumé.example.org would lead to an IRI of + http://validator.w3.org/check?uri=http%3A%2F%2Frésumé. + example.org, which would convert to a URI of + http://validator.w3.org/check?uri=http%3A%2F%2Fr%C3%A9sum%C3%A9. + example.org. The server side implementation would be responsible + for making the necessary conversions to be able to retrieve the + Web page. + + Systems accepting IRIs MAY also deal with the printable characters in + US-ASCII that are not allowed in URIs, namely "<", ">", '"', space, + "{", "}", "|", "\", "^", and "`", in step 2 above. If these + characters are found but are not converted, then the conversion + SHOULD fail. Please note that the number sign ("#"), the percent + sign ("%"), and the square bracket characters ("[", "]") are not part + of the above list and MUST NOT be converted. Protocols and formats + that have used earlier definitions of IRIs including these characters + MAY require percent-encoding of these characters as a preprocessing + step to extract the actual IRI from a given field. This + preprocessing MAY also be used by applications allowing the user to + enter an IRI. + + Note: In this process (in step 2.3), characters allowed in URI + references and existing percent-encoded sequences are not encoded + further. (This mapping is similar to, but different from, the + encoding applied when arbitrary content is included in some part + of a URI.) For example, an IRI of + "http://www.example.org/red%09rosé#red" (in XML notation) is + converted to + "http://www.example.org/red%09ros%C3%A9#red", not to something + like + "http%3A%2F%2Fwww.example.org%2Fred%2509ros%C3%A9%23red". + + Note: Some older software transcoding to UTF-8 may produce illegal + output for some input, in particular for characters outside the + BMP (Basic Multilingual Plane). As an example, for the IRI with + non-BMP characters (in XML Notation): + "http://example.com/𐌀𐌁𐌂"; + + + +Duerst & Suignard Standards Track [Page 13] + +RFC 3987 Internationalized Resource Identifiers January 2005 + + + which contains the first three letters of the Old Italic alphabet, + the correct conversion to a URI is + "http://example.com/%F0%90%8C%80%F0%90%8C%81%F0%90%8C%82" + +3.2. Converting URIs to IRIs + + In some situations, converting a URI into an equivalent IRI may be + desirable. This section gives a procedure for this conversion. The + conversion described in this section will always result in an IRI + that maps back to the URI used as an input for the conversion (except + for potential case differences in percent-encoding and for potential + percent-encoded unreserved characters). However, the IRI resulting + from this conversion may not be exactly the same as the original IRI + (if there ever was one). + + URI-to-IRI conversion removes percent-encodings, but not all + percent-encodings can be eliminated. There are several reasons for + this: + + 1. Some percent-encodings are necessary to distinguish percent- + encoded and unencoded uses of reserved characters. + + 2. Some percent-encodings cannot be interpreted as sequences of + UTF-8 octets. + + (Note: The octet patterns of UTF-8 are highly regular. + Therefore, there is a very high probability, but no guarantee, + that percent-encodings that can be interpreted as sequences of + UTF-8 octets actually originated from UTF-8. For a detailed + discussion, see [Duerst97].) + + 3. The conversion may result in a character that is not appropriate + in an IRI. See sections 2.2, 4.1, and 6.1 for further details. + + Conversion from a URI to an IRI is done by using the following steps + (or any other algorithm that produces the same result): + + 1. Represent the URI as a sequence of octets in US-ASCII. + + 2. Convert all percent-encodings ("%" followed by two hexadecimal + digits) to the corresponding octets, except those corresponding + to "%", characters in "reserved", and characters in US-ASCII not + allowed in URIs. + + 3. Re-percent-encode any octet produced in step 2 that is not part + of a strictly legal UTF-8 octet sequence. + + + + + +Duerst & Suignard Standards Track [Page 14] + +RFC 3987 Internationalized Resource Identifiers January 2005 + + + 4. Re-percent-encode all octets produced in step 3 that in UTF-8 + represent characters that are not appropriate according to + sections 2.2, 4.1, and 6.1. + + 5. Interpret the resulting octet sequence as a sequence of characters + encoded in UTF-8. + + This procedure will convert as many percent-encoded characters as + possible to characters in an IRI. Because there are some choices + when step 4 is applied (see section 6.1), results may vary. + + Conversions from URIs to IRIs MUST NOT use any character encoding + other than UTF-8 in steps 3 and 4, even if it might be possible to + guess from the context that another character encoding than UTF-8 was + used in the URI. For example, the URI + "http://www.example.org/r%E9sum%E9.html" might with some guessing be + interpreted to contain two e-acute characters encoded as iso-8859-1. + It must not be converted to an IRI containing these e-acute + characters. Otherwise, in the future the IRI will be mapped to + "http://www.example.org/r%C3%A9sum%C3%A9.html", which is a different + URI from "http://www.example.org/r%E9sum%E9.html". + +3.2.1. Examples + + This section shows various examples of converting URIs to IRIs. Each + example shows the result after each of the steps 1 through 5 is + applied. XML Notation is used for the final result. Octets are + denoted by "<" followed by two hexadecimal digits followed by ">". + + The following example contains the sequence "%C3%BC", which is a + strictly legal UTF-8 sequence, and which is converted into the actual + character U+00FC, LATIN SMALL LETTER U WITH DIAERESIS (also known as + u-umlaut). + + 1. http://www.example.org/D%C3%BCrst + + 2. http://www.example.org/Drst + + 3. http://www.example.org/Drst + + 4. http://www.example.org/Drst + + 5. http://www.example.org/Dürst + + The following example contains the sequence "%FC", which might + represent U+00FC, LATIN SMALL LETTER U WITH DIAERESIS, in the + iso-8859-1 character encoding. (It might represent other characters + in other character encodings. For example, the octet in + + + +Duerst & Suignard Standards Track [Page 15] + +RFC 3987 Internationalized Resource Identifiers January 2005 + + + iso-8859-5 represents U+045C, CYRILLIC SMALL LETTER KJE.) Because + is not part of a strictly legal UTF-8 sequence, it is + re-percent-encoded in step 3. + + 1. http://www.example.org/D%FCrst + + 2. http://www.example.org/Drst + + 3. http://www.example.org/D%FCrst + + 4. http://www.example.org/D%FCrst + + 5. http://www.example.org/D%FCrst + + The following example contains "%e2%80%ae", which is the percent- + encoded UTF-8 character encoding of U+202E, RIGHT-TO-LEFT OVERRIDE. + Section 4.1 forbids the direct use of this character in an IRI. + Therefore, the corresponding octets are re-percent-encoded in step 4. + This example shows that the case (upper- or lowercase) of letters + used in percent-encodings may not be preserved. The example also + contains a punycode-encoded domain name label (xn--99zt52a), which is + not converted. + + 1. http://xn--99zt52a.example.org/%e2%80%ae + + 2. http://xn--99zt52a.example.org/<80> + + 3. http://xn--99zt52a.example.org/<80> + + 4. http://xn--99zt52a.example.org/%E2%80%AE + + 5. http://xn--99zt52a.example.org/%E2%80%AE + + Implementations with scheme-specific knowledge MAY convert + punycode-encoded domain name labels to the corresponding characters + by using the ToUnicode procedure. Thus, for the example above, the + label "xn--99zt52a" may be converted to U+7D0D U+8C46 (Japanese + Natto), leading to the overall IRI of + "http://納豆.example.org/%E2%80%AE". + +4. Bidirectional IRIs for Right-to-Left Languages + + Some UCS characters, such as those used in the Arabic and Hebrew + scripts, have an inherent right-to-left (rtl) writing direction. + IRIs containing these characters (called bidirectional IRIs or Bidi + IRIs) require additional attention because of the non-trivial + + + + + +Duerst & Suignard Standards Track [Page 16] + +RFC 3987 Internationalized Resource Identifiers January 2005 + + + relation between logical representation (used for digital + representation and for reading/spelling) and visual representation + (used for display/printing). + + Because of the complex interaction between the logical + representation, the visual representation, and the syntax of a Bidi + IRI, a balance is needed between various requirements. The main + requirements are + + 1. user-predictable conversion between visual and logical + representation; + + 2. the ability to include a wide range of characters in various + parts of the IRI; and + + 3. minor or no changes or restrictions for implementations. + +4.1. Logical Storage and Visual Presentation + + When stored or transmitted in digital representation, bidirectional + IRIs MUST be in full logical order and MUST conform to the IRI syntax + rules (which includes the rules relevant to their scheme). This + ensures that bidirectional IRIs can be processed in the same way as + other IRIs. + + Bidirectional IRIs MUST be rendered by using the Unicode + Bidirectional Algorithm [UNIV4], [UNI9]. Bidirectional IRIs MUST be + rendered in the same way as they would be if they were in a + left-to-right embedding; i.e., as if they were preceded by U+202A, + LEFT-TO-RIGHT EMBEDDING (LRE), and followed by U+202C, POP + DIRECTIONAL FORMATTING (PDF). Setting the embedding direction can + also be done in a higher-level protocol (e.g., the dir='ltr' + attribute in HTML). + + There is no requirement to use the above embedding if the display is + still the same without the embedding. For example, a bidirectional + IRI in a text with left-to-right base directionality (such as used + for English or Cyrillic) that is preceded and followed by whitespace + and strong left-to-right characters does not need an embedding. + Also, a bidirectional relative IRI reference that only contains + strong right-to-left characters and weak characters and that starts + and ends with a strong right-to-left character and appears in a text + with right-to-left base directionality (such as used for Arabic or + Hebrew) and is preceded and followed by whitespace and strong + characters does not need an embedding. + + + + + + +Duerst & Suignard Standards Track [Page 17] + +RFC 3987 Internationalized Resource Identifiers January 2005 + + + In some other cases, using U+200E, LEFT-TO-RIGHT MARK (LRM), may be + sufficient to force the correct display behavior. However, the + details of the Unicode Bidirectional algorithm are not always easy to + understand. Implementers are strongly advised to err on the side of + caution and to use embedding in all cases where they are not + completely sure that the display behavior is unaffected without the + embedding. + + The Unicode Bidirectional Algorithm ([UNI9], section 4.3) permits + higher-level protocols to influence bidirectional rendering. Such + changes by higher-level protocols MUST NOT be used if they change the + rendering of IRIs. + + The bidirectional formatting characters that may be used before or + after the IRI to ensure correct display are not themselves part of + the IRI. IRIs MUST NOT contain bidirectional formatting characters + (LRM, RLM, LRE, RLE, LRO, RLO, and PDF). They affect the visual + rendering of the IRI but do not appear themselves. It would + therefore not be possible to input an IRI with such characters + correctly. + +4.2. Bidi IRI Structure + + The Unicode Bidirectional Algorithm is designed mainly for running + text. To make sure that it does not affect the rendering of + bidirectional IRIs too much, some restrictions on bidirectional IRIs + are necessary. These restrictions are given in terms of delimiters + (structural characters, mostly punctuation such as "@", ".", ":", and + "/") and components (usually consisting mostly of letters and + digits). + + The following syntax rules from section 2.2 correspond to components + for the purpose of Bidi behavior: iuserinfo, ireg-name, isegment, + isegment-nz, isegment-nz-nc, ireg-name, iquery, and ifragment. + + Specifications that define the syntax of any of the above components + MAY divide them further and define smaller parts to be components + according to this document. As an example, the restrictions of + [RFC3490] on bidirectional domain names correspond to treating each + label of a domain name as a component for schemes with ireg-name as a + domain name. Even where the components are not defined formally, it + may be helpful to think about some syntax in terms of components and + to apply the relevant restrictions. For example, for the usual + name/value syntax in query parts, it is convenient to treat each name + and each value as a component. As another example, the extensions in + a resource name can be treated as separate components. + + + + + +Duerst & Suignard Standards Track [Page 18] + +RFC 3987 Internationalized Resource Identifiers January 2005 + + + For each component, the following restrictions apply: + + 1. A component SHOULD NOT use both right-to-left and left-to-right + characters. + + 2. A component using right-to-left characters SHOULD start and end + with right-to-left characters. + + The above restrictions are given as shoulds, rather than as musts. + For IRIs that are never presented visually, they are not relevant. + However, for IRIs in general, they are very important to ensure + consistent conversion between visual presentation and logical + representation, in both directions. + + Note: In some components, the above restrictions may actually be + strictly enforced. For example, [RFC3490] requires that these + restrictions apply to the labels of a host name for those schemes + where ireg-name is a host name. In some other components (for + example, path components) following these restrictions may not be + too difficult. For other components, such as parts of the query + part, it may be very difficult to enforce the restrictions because + the values of query parameters may be arbitrary character + sequences. + + If the above restrictions cannot be satisfied otherwise, the affected + component can always be mapped to URI notation as described in + section 3.1. Please note that the whole component has to be mapped + (see also Example 9 below). + +4.3. Input of Bidi IRIs + + Bidi input methods MUST generate Bidi IRIs in logical order while + rendering them according to section 4.1. During input, rendering + SHOULD be updated after every new character is input to avoid end- + user confusion. + +4.4. Examples + + This section gives examples of bidirectional IRIs, in Bidi Notation. + It shows legal IRIs with the relationship between logical and visual + representation and explains how certain phenomena in this + relationship may look strange to somebody not familiar with + bidirectional behavior, but familiar to users of Arabic and Hebrew. + It also shows what happens if the restrictions given in section 4.2 + are not followed. The examples below can be seen at [BidiEx], in + Arabic, Hebrew, and Bidi Notation variants. + + + + + +Duerst & Suignard Standards Track [Page 19] + +RFC 3987 Internationalized Resource Identifiers January 2005 + + + To read the bidi text in the examples, read the visual representation + from left to right until you encounter a block of rtl text. Read the + rtl block (including slashes and other special characters) from right + to left, then continue at the next unread ltr character. + + Example 1: A single component with rtl characters is inverted: + Logical representation: "http://ab.CDEFGH.ij/kl/mn/op.html" + Visual representation: "http://ab.HGFEDC.ij/kl/mn/op.html" + Components can be read one by one, and each component can be read in + its natural direction. + + Example 2: More than one consecutive component with rtl characters is + inverted as a whole: + Logical representation: "http://ab.CDE.FGH/ij/kl/mn/op.html" + Visual representation: "http://ab.HGF.EDC/ij/kl/mn/op.html" + A sequence of rtl components is read rtl, in the same way as a + sequence of rtl words is read rtl in a bidi text. + + Example 3: All components of an IRI (except for the scheme) are rtl. + All rtl components are inverted overall: + Logical representation: "http://AB.CD.EF/GH/IJ/KL?MN=OP;QR=ST#UV" + Visual representation: "http://VU#TS=RQ;PO=NM?LK/JI/HG/FE.DC.BA" + The whole IRI (except the scheme) is read rtl. Delimiters between + rtl components stay between the respective components; delimiters + between ltr and rtl components don't move. + + Example 4: Each of several sequences of rtl components is inverted on + its own: + Logical representation: "http://AB.CD.ef/gh/IJ/KL.html" + Visual representation: "http://DC.BA.ef/gh/LK/JI.html" + Each sequence of rtl components is read rtl, in the same way as each + sequence of rtl words in an ltr text is read rtl. + + Example 5: Example 2, applied to components of different kinds: + Logical representation: "http://ab.cd.EF/GH/ij/kl.html" + Visual representation: "http://ab.cd.HG/FE/ij/kl.html" + The inversion of the domain name label and the path component may be + unexpected, but it is consistent with other bidi behavior. For + reassurance that the domain component really is "ab.cd.EF", it may be + helpful to read aloud the visual representation following the bidi + algorithm. After "http://ab.cd." one reads the RTL block + "E-F-slash-G-H", which corresponds to the logical representation. + + Example 6: Same as Example 5, with more rtl components: + Logical representation: "http://ab.CD.EF/GH/IJ/kl.html" + Visual representation: "http://ab.JI/HG/FE.DC/kl.html" + The inversion of the domain name labels and the path components may + be easier to identify because the delimiters also move. + + + +Duerst & Suignard Standards Track [Page 20] + +RFC 3987 Internationalized Resource Identifiers January 2005 + + + Example 7: A single rtl component includes digits: + Logical representation: "http://ab.CDE123FGH.ij/kl/mn/op.html" + Visual representation: "http://ab.HGF123EDC.ij/kl/mn/op.html" + Numbers are written ltr in all cases but are treated as an additional + embedding inside a run of rtl characters. This is completely + consistent with usual bidirectional text. + + Example 8 (not allowed): Numbers are at the start or end of an rtl + component: + Logical representation: "http://ab.cd.ef/GH1/2IJ/KL.html" + Visual representation: "http://ab.cd.ef/LK/JI1/2HG.html" + The sequence "1/2" is interpreted by the bidi algorithm as a + fraction, fragmenting the components and leading to confusion. There + are other characters that are interpreted in a special way close to + numbers; in particular, "+", "-", "#", "$", "%", ",", ".", and ":". + + Example 9 (not allowed): The numbers in the previous example are + percent-encoded: + Logical representation: "http://ab.cd.ef/GH%31/%32IJ/KL.html", + Visual representation (Hebrew): "http://ab.cd.ef/%31HG/LK/JI%32.html" + Visual representation (Arabic): "http://ab.cd.ef/31%HG/%LK/JI32.html" + Depending on whether the uppercase letters represent Arabic or + Hebrew, the visual representation is different. + + Example 10 (allowed but not recommended): + Logical representation: "http://ab.CDEFGH.123/kl/mn/op.html" + Visual representation: "http://ab.123.HGFEDC/kl/mn/op.html" + Components consisting of only numbers are allowed (it would be rather + difficult to prohibit them), but these may interact with adjacent RTL + components in ways that are not easy to predict. + +5. Normalization and Comparison + + Note: The structure and much of the material for this section is + taken from section 6 of [RFC3986]; the differences are due to the + specifics of IRIs. + + One of the most common operations on IRIs is simple comparison: + Determining whether two IRIs are equivalent without using the IRIs or + the mapped URIs to access their respective resource(s). A comparison + is performed whenever a response cache is accessed, a browser checks + its history to color a link, or an XML parser processes tags within a + namespace. Extensive normalization prior to comparison of IRIs may + be used by spiders and indexing engines to prune a search space or + reduce duplication of request actions and response storage. + + + + + + +Duerst & Suignard Standards Track [Page 21] + +RFC 3987 Internationalized Resource Identifiers January 2005 + + + IRI comparison is performed for some particular purpose. Protocols + or implementations that compare IRIs for different purposes will + often be subject to differing design trade-offs in regards to how + much effort should be spent in reducing aliased identifiers. This + section describes various methods that may be used to compare IRIs, + the trade-offs between them, and the types of applications that might + use them. + +5.1. Equivalence + + Because IRIs exist to identify resources, presumably they should be + considered equivalent when they identify the same resource. However, + this definition of equivalence is not of much practical use, as there + is no way for an implementation to compare two resources unless it + has full knowledge or control of them. For this reason, determination + of equivalence or difference of IRIs is based on string comparison, + perhaps augmented by reference to additional rules provided by URI + scheme definitions. We use the terms "different" and "equivalent" to + describe the possible outcomes of such comparisons, but there are + many application-dependent versions of equivalence. + + Even though it is possible to determine that two IRIs are equivalent, + IRI comparison is not sufficient to determine whether two IRIs + identify different resources. For example, an owner of two different + domain names could decide to serve the same resource from both, + resulting in two different IRIs. Therefore, comparison methods are + designed to minimize false negatives while strictly avoiding false + positives. + + In testing for equivalence, applications should not directly compare + relative references; the references should be converted to their + respective target IRIs before comparison. When IRIs are compared to + select (or avoid) a network action, such as retrieval of a + representation, fragment components (if any) should be excluded from + the comparison. + + Applications using IRIs as identity tokens with no relationship to a + protocol MUST use the Simple String Comparison (see section 5.3.1). + All other applications MUST select one of the comparison practices + from the Comparison Ladder (see section 5.3 or, after IRI-to-URI + conversion, select one of the comparison practices from the URI + comparison ladder in [RFC3986], section 6.2) + +5.2. Preparation for Comparison + + Any kind of IRI comparison REQUIRES that all escapings or encodings + in the protocol or format that carries an IRI are resolved. This is + usually done when the protocol or format is parsed. Examples of such + + + +Duerst & Suignard Standards Track [Page 22] + +RFC 3987 Internationalized Resource Identifiers January 2005 + + + escapings or encodings are entities and numeric character references + in [HTML4] and [XML1]. As an example, + "http://example.org/rosé" (in HTML), + "http://example.org/rosé"; (in HTML or XML), and + "http://example.org/rosé"; (in HTML or XML) are all resolved into + what is denoted in this document (see section 1.4) as + "http://example.org/rosé"; (the "é" here standing for the + actual e-acute character, to compensate for the fact that this + document cannot contain non-ASCII characters). + + Similar considerations apply to encodings such as Transfer Codings in + HTTP (see [RFC2616]) and Content Transfer Encodings in MIME + ([RFC2045]), although in these cases, the encoding is based not on + characters but on octets, and additional care is required to make + sure that characters, and not just arbitrary octets, are compared + (see section 5.3.1). + +5.3. Comparison Ladder + + In practice, a variety of methods are used, to test IRI equivalence. + These methods fall into a range distinguished by the amount of + processing required and the degree to which the probability of false + negatives is reduced. As noted above, false negatives cannot be + eliminated. In practice, their probability can be reduced, but this + reduction requires more processing and is not cost-effective for all + applications. + + If this range of comparison practices is considered as a ladder, the + following discussion will climb the ladder, starting with practices + that are cheap but have a relatively higher chance of producing false + negatives, and proceeding to those that have higher computational + cost and lower risk of false negatives. + +5.3.1. Simple String Comparison + + If two IRIs, when considered as character strings, are identical, + then it is safe to conclude that they are equivalent. This type of + equivalence test has very low computational cost and is in wide use + in a variety of applications, particularly in the domain of parsing. + It is also used when a definitive answer to the question of IRI + equivalence is needed that is independent of the scheme used and that + can be calculated quickly and without accessing a network. An + example of such a case is XML Namespaces ([XMLNamespace]). + + Testing strings for equivalence requires some basic precautions. This + procedure is often referred to as "bit-for-bit" or "byte-for-byte" + comparison, which is potentially misleading. Testing strings for + equality is normally based on pair comparison of the characters that + + + +Duerst & Suignard Standards Track [Page 23] + +RFC 3987 Internationalized Resource Identifiers January 2005 + + + make up the strings, starting from the first and proceeding until + both strings are exhausted and all characters are found to be equal, + until a pair of characters compares unequal, or until one of the + strings is exhausted before the other. + + This character comparison requires that each pair of characters be + put in comparable encoding form. For example, should one IRI be + stored in a byte array in UTF-8 encoding form and the second in a + UTF-16 encoding form, bit-for-bit comparisons applied naively will + produce errors. It is better to speak of equality on a + character-for-character rather than on a byte-for-byte or bit-for-bit + basis. In practical terms, character-by-character comparisons should + be done codepoint by codepoint after conversion to a common character + encoding form. When comparing character by character, the comparison + function MUST NOT map IRIs to URIs, because such a mapping would + create additional spurious equivalences. It follows that an IRI + SHOULD NOT be modified when being transported if there is any chance + that this IRI might be used as an identifier. + + False negatives are caused by the production and use of IRI aliases. + Unnecessary aliases can be reduced, regardless of the comparison + method, by consistently providing IRI references in an already + normalized form (i.e., a form identical to what would be produced + after normalization is applied, as described below). Protocols and + data formats often limit some IRI comparisons to simple string + comparison, based on the theory that people and implementations will, + in their own best interest, be consistent in providing IRI + references, or at least be consistent enough to negate any efficiency + that might be obtained from further normalization. + +5.3.2. Syntax-Based Normalization + + Implementations may use logic based on the definitions provided by + this specification to reduce the probability of false negatives. This + processing is moderately higher in cost than character-for-character + string comparison. For example, an application using this approach + could reasonably consider the following two IRIs equivalent: + + example://a/b/c/%7Bfoo%7D/rosé + eXAMPLE://a/./b/../b/%63/%7bfoo%7d/ros%C3%A9 + + Web user agents, such as browsers, typically apply this type of IRI + normalization when determining whether a cached response is + available. Syntax-based normalization includes such techniques as + case normalization, character normalization, percent-encoding + normalization, and removal of dot-segments. + + + + + +Duerst & Suignard Standards Track [Page 24] + +RFC 3987 Internationalized Resource Identifiers January 2005 + + +5.3.2.1. Case Normalization + + For all IRIs, the hexadecimal digits within a percent-encoding + triplet (e.g., "%3a" versus "%3A") are case-insensitive and therefore + should be normalized to use uppercase letters for the digits A - F. + + When an IRI uses components of the generic syntax, the component + syntax equivalence rules always apply; namely, that the scheme and + US-ASCII only host are case insensitive and therefore should be + normalized to lowercase. For example, the URI + "HTTP://www.EXAMPLE.com/" is equivalent to "http://www.example.com/". + Case equivalence for non-ASCII characters in IRI components that are + IDNs are discussed in section 5.3.3. The other generic syntax + components are assumed to be case sensitive unless specifically + defined otherwise by the scheme. + + Creating schemes that allow case-insensitive syntax components + containing non-ASCII characters should be avoided. Case normalization + of non-ASCII characters can be culturally dependent and is always a + complex operation. The only exception concerns non-ASCII host names + for which the character normalization includes a mapping step derived + from case folding. + +5.3.2.2. Character Normalization + + The Unicode Standard [UNIV4] defines various equivalences between + sequences of characters for various purposes. Unicode Standard Annex + #15 [UTR15] defines various Normalization Forms for these + equivalences, in particular Normalization Form C (NFC, Canonical + Decomposition, followed by Canonical Composition) and Normalization + Form KC (NFKC, Compatibility Decomposition, followed by Canonical + Composition). + + Equivalence of IRIs MUST rely on the assumption that IRIs are + appropriately pre-character-normalized rather than apply character + normalization when comparing two IRIs. The exceptions are conversion + from a non-digital form, and conversion from a non-UCS-based + character encoding to a UCS-based character encoding. In these cases, + NFC or a normalizing transcoder using NFC MUST be used for + interoperability. To avoid false negatives and problems with + transcoding, IRIs SHOULD be created by using NFC. Using NFKC may + avoid even more problems; for example, by choosing half-width Latin + letters instead of full-width ones, and full-width instead of + half-width Katakana. + + As an example, "http://www.example.org/résumé.html" (in XML + Notation) is in NFC. On the other hand, + "http://www.example.org/résumé.html" is not in NFC. + + + +Duerst & Suignard Standards Track [Page 25] + +RFC 3987 Internationalized Resource Identifiers January 2005 + + + The former uses precombined e-acute characters, and the latter uses + "e" characters followed by combining acute accents. Both usages are + defined as canonically equivalent in [UNIV4]. + + Note: Because it is unknown how a particular sequence of characters + is being treated with respect to character normalization, it would + be inappropriate to allow third parties to normalize an IRI + arbitrarily. This does not contradict the recommendation that + when a resource is created, its IRI should be as character + normalized as possible (i.e., NFC or even NFKC). This is similar + to the uppercase/lowercase problems. Some parts of a URI are case + insensitive (domain name). For others, it is unclear whether they + are case sensitive, case insensitive, or something in between + (e.g., case sensitive, but with a multiple choice selection if the + wrong case is used, instead of a direct negative result). The + best recipe is that the creator use a reasonable capitalization + and, when transferring the URI, capitalization never be changed. + + Various IRI schemes may allow the usage of Internationalized Domain + Names (IDN) [RFC3490] either in the ireg-name part or elsewhere. + Character Normalization also applies to IDNs, as discussed in section + 5.3.3. + +5.3.2.3. Percent-Encoding Normalization + + The percent-encoding mechanism (section 2.1 of [RFC3986]) is a + frequent source of variance among otherwise identical IRIs. In + addition to the case normalization issue noted above, some IRI + producers percent-encode octets that do not require percent-encoding, + resulting in IRIs that are equivalent to their non encoded + counterparts. These IRIs should be normalized by decoding any + percent-encoded octet sequence that corresponds to an unreserved + character, as described in section 2.3 of [RFC3986]. + + For actual resolution, differences in percent-encoding (except for + the percent-encoding of reserved characters) MUST always result in + the same resource. For example, "http://example.org/~user", + "http://example.org/%7euser", and "http://example.org/%7Euser", must + resolve to the same resource. + + If this kind of equivalence is to be tested, the percent-encoding of + both IRIs to be compared has to be aligned; for example, by + converting both IRIs to URIs (see section 3.1), eliminating escape + differences in the resulting URIs, and making sure that the case of + the hexadecimal characters in the percent-encoding is always the same + (preferably uppercase). If the IRI is to be passed to another + + + + + +Duerst & Suignard Standards Track [Page 26] + +RFC 3987 Internationalized Resource Identifiers January 2005 + + + application or used further in some other way, its original form MUST + be preserved. The conversion described here should be performed only + for local comparison. + +5.3.2.4. Path Segment Normalization + + The complete path segments "." and ".." are intended only for use + within relative references (section 4.1 of [RFC3986]) and are removed + as part of the reference resolution process (section 5.2 of + [RFC3986]). However, some implementations may incorrectly assume + that reference resolution is not necessary when the reference is + already an IRI, and thus fail to remove dot-segments when they occur + in non-relative paths. IRI normalizers should remove dot-segments by + applying the remove_dot_segments algorithm to the path, as described + in section 5.2.4 of [RFC3986]. + +5.3.3. Scheme-Based Normalization + + The syntax and semantics of IRIs vary from scheme to scheme, as + described by the defining specification for each scheme. + Implementations may use scheme-specific rules, at further processing + cost, to reduce the probability of false negatives. For example, + because the "http" scheme makes use of an authority component, has a + default port of "80", and defines an empty path to be equivalent to + "/", the following four IRIs are equivalent: + + http://example.com + http://example.com/ + http://example.com:/ + http://example.com:80/ + + In general, an IRI that uses the generic syntax for authority with an + empty path should be normalized to a path of "/". Likewise, an + explicit ":port", for which the port is empty or the default for the + scheme, is equivalent to one where the port and its ":" delimiter are + elided and thus should be removed by scheme-based normalization. For + example, the second IRI above is the normal form for the "http" + scheme. + + Another case where normalization varies by scheme is in the handling + of an empty authority component or empty host subcomponent. For many + scheme specifications, an empty authority or host is considered an + error; for others, it is considered equivalent to "localhost" or the + end-user's host. When a scheme defines a default for authority and + an IRI reference to that default is desired, the reference should be + normalized to an empty authority for the sake of uniformity, brevity, + + + + + +Duerst & Suignard Standards Track [Page 27] + +RFC 3987 Internationalized Resource Identifiers January 2005 + + + and internationalization. If, however, either the userinfo or port + subcomponents are non-empty, then the host should be given explicitly + even if it matches the default. + + Normalization should not remove delimiters when their associated + component is empty unless it is licensed to do so by the scheme + specification. For example, the IRI "http://example.com/?" cannot be + assumed to be equivalent to any of the examples above. Likewise, the + presence or absence of delimiters within a userinfo subcomponent is + usually significant to its interpretation. The fragment component is + not subject to any scheme-based normalization; thus, two IRIs that + differ only by the suffix "#" are considered different regardless of + the scheme. + + Some IRI schemes may allow the usage of Internationalized Domain + Names (IDN) [RFC3490] either in their ireg-name part or elsewhere. + When in use in IRIs, those names SHOULD be validated by using the + ToASCII operation defined in [RFC3490], with the flags + "UseSTD3ASCIIRules" and "AllowUnassigned". An IRI containing an + invalid IDN cannot successfully be resolved. Validated IDN + components of IRIs SHOULD be character normalized by using the + Nameprep process [RFC3491]; however, for legibility purposes, they + SHOULD NOT be converted into ASCII Compatible Encoding (ACE). + + Scheme-based normalization may also consider IDN components and their + conversions to punycode as equivalent. As an example, + "http://résumé.example.org" may be considered equivalent to + "http://xn--rsum-bpad.example.org". + + Other scheme-specific normalizations are possible. + +5.3.4. Protocol-Based Normalization + + Substantial effort to reduce the incidence of false negatives is + often cost-effective for web spiders. Consequently, they implement + even more aggressive techniques in IRI comparison. For example, if + they observe that an IRI such as + + http://example.com/data + + redirects to an IRI differing only in the trailing slash + + http://example.com/data/ + + they will likely regard the two as equivalent in the future. This + kind of technique is only appropriate when equivalence is clearly + indicated by both the result of accessing the resources and the + + + + +Duerst & Suignard Standards Track [Page 28] + +RFC 3987 Internationalized Resource Identifiers January 2005 + + + common conventions of their scheme's dereference algorithm (in this + case, use of redirection by HTTP origin servers to avoid problems + with relative references). + +6. Use of IRIs + +6.1. Limitations on UCS Characters Allowed in IRIs + + This section discusses limitations on characters and character + sequences usable for IRIs beyond those given in section 2.2 and + section 4.1. The considerations in this section are relevant when + IRIs are created and when URIs are converted to IRIs. + + a. The repertoire of characters allowed in each IRI component is + limited by the definition of that component. For example, the + definition of the scheme component does not allow characters + beyond US-ASCII. + + (Note: In accordance with URI practice, generic IRI software + cannot and should not check for such limitations.) + + b. The UCS contains many areas of characters for which there are + strong visual look-alikes. Because of the likelihood of + transcription errors, these also should be avoided. This + includes the full-width equivalents of Latin characters, + half-width Katakana characters for Japanese, and many others. It + also includes many look-alikes of "space", "delims", and + "unwise", characters excluded in [RFC3491]. + + Additional information is available from [UNIXML]. [UNIXML] is + written in the context of running text rather than in that of + identifiers. Nevertheless, it discusses many of the categories of + characters not appropriate for IRIs. + +6.2. Software Interfaces and Protocols + + Although an IRI is defined as a sequence of characters, software + interfaces for URIs typically function on sequences of octets or + other kinds of code units. Thus, software interfaces and protocols + MUST define which character encoding is used. + + Intermediate software interfaces between IRI-capable components and + URI-only components MUST map the IRIs per section 3.1, when + transferring from IRI-capable to URI-only components. This mapping + SHOULD be applied as late as possible. It SHOULD NOT be applied + between components that are known to be able to handle IRIs. + + + + + +Duerst & Suignard Standards Track [Page 29] + +RFC 3987 Internationalized Resource Identifiers January 2005 + + +6.3. Format of URIs and IRIs in Documents and Protocols + + Document formats that transport URIs may have to be upgraded to allow + the transport of IRIs. In cases where the document as a whole has a + native character encoding, IRIs MUST also be encoded in this + character encoding and converted accordingly by a parser or + interpreter. IRI characters not expressible in the native character + encoding SHOULD be escaped by using the escaping conventions of the + document format if such conventions are available. Alternatively, + they MAY be percent-encoded according to section 3.1. For example, in + HTML or XML, numeric character references SHOULD be used. If a + document as a whole has a native character encoding and that + character encoding is not UTF-8, then IRIs MUST NOT be placed into + the document in the UTF-8 character encoding. + + Note: Some formats already accommodate IRIs, although they use + different terminology. HTML 4.0 [HTML4] defines the conversion from + IRIs to URIs as error-avoiding behavior. XML 1.0 [XML1], XLink + [XLink], XML Schema [XMLSchema], and specifications based upon them + allow IRIs. Also, it is expected that all relevant new W3C formats + and protocols will be required to handle IRIs [CharMod]. + +6.4. Use of UTF-8 for Encoding Original Characters + + This section discusses details and gives examples for point c) in + section 1.2. To be able to use IRIs, the URI corresponding to the + IRI in question has to encode original characters into octets by + using UTF-8. This can be specified for all URIs of a URI scheme or + can apply to individual URIs for schemes that do not specify how to + encode original characters. It can apply to the whole URI, or only + to some part. For background information on encoding characters into + URIs, see also section 2.5 of [RFC3986]. + + For new URI schemes, using UTF-8 is recommended in [RFC2718]. + Examples where UTF-8 is already used are the URN syntax [RFC2141], + IMAP URLs [RFC2192], and POP URLs [RFC2384]. On the other hand, + because the HTTP URL scheme does not specify how to encode original + characters, only some HTTP URLs can have corresponding but different + IRIs. + + For example, for a document with a URI of + "http://www.example.org/r%C3%A9sum%C3%A9.html", it is possible to + construct a corresponding IRI (in XML notation, see, section 1.4): + "http://www.example.org/résumé.html" ("é"; stands for + the e-acute character, and "%C3%A9" is the UTF-8 encoded and + percent-encoded representation of that character). On the other + hand, for a document with a URI of + + + + +Duerst & Suignard Standards Track [Page 30] + +RFC 3987 Internationalized Resource Identifiers January 2005 + + + "http://www.example.org/r%E9sum%E9.html", the percent-encoding octets + cannot be converted to actual characters in an IRI, as the + percent-encoding is not based on UTF-8. + + This means that for most URI schemes, there is no need to upgrade + their scheme definition in order for them to work with IRIs. The + main case where upgrading makes sense is when a scheme definition, or + a particular component of a scheme, is strictly limited to the use of + US-ASCII characters with no provision to include non-ASCII + characters/octets via percent-encoding, or if a scheme definition + currently uses highly scheme-specific provisions for the encoding of + non-ASCII characters. An example of this is the mailto: scheme + [RFC2368]. + + This specification does not upgrade any scheme specifications in any + way; this has to be done separately. Also, note that there is no + such thing as an "IRI scheme"; all IRIs use URI schemes, and all URI + schemes can be used with IRIs, even though in some cases only by + using URIs directly as IRIs, without any conversion. + + URI schemes can impose restrictions on the syntax of scheme-specific + URIs; i.e., URIs that are admissible under the generic URI syntax + [RFC3986] may not be admissible due to narrower syntactic constraints + imposed by a URI scheme specification. URI scheme definitions cannot + broaden the syntactic restrictions of the generic URI syntax; + otherwise, it would be possible to generate URIs that satisfied the + scheme-specific syntactic constraints without satisfying the + syntactic constraints of the generic URI syntax. However, additional + syntactic constraints imposed by URI scheme specifications are + applicable to IRI, as the corresponding URI resulting from the + mapping defined in section 3.1 MUST be a valid URI under the + syntactic restrictions of generic URI syntax and any narrower + restrictions imposed by the corresponding URI scheme specification. + + The requirement for the use of UTF-8 applies to all parts of a URI + (with the potential exception of the ireg-name part; see section + 3.1). However, it is possible that the capability of IRIs to + represent a wide range of characters directly is used just in some + parts of the IRI (or IRI reference). The other parts of the IRI may + only contain US-ASCII characters, or they may not be based on UTF-8. + They may be based on another character encoding, or they may directly + encode raw binary data (see also [RFC2397]). + + For example, it is possible to have a URI reference of + "http://www.example.org/r%E9sum%E9.xml#r%C3%A9sum%C3%A9", where the + document name is encoded in iso-8859-1 based on server settings, but + where the fragment identifier is encoded in UTF-8 according to + + + + +Duerst & Suignard Standards Track [Page 31] + +RFC 3987 Internationalized Resource Identifiers January 2005 + + + [XPointer]. The IRI corresponding to the above URI would be (in XML + notation) + "http://www.example.org/r%E9sum%E9.xml#résumé";. + + Similar considerations apply to query parts. The functionality of + IRIs (namely, to be able to include non-ASCII characters) can only be + used if the query part is encoded in UTF-8. + +6.5. Relative IRI References + + Processing of relative IRI references against a base is handled + straightforwardly; the algorithms of [RFC3986] can be applied + directly, treating the characters additionally allowed in IRI + references in the same way that unreserved characters are in URI + references. + +7. URI/IRI Processing Guidelines (Informative) + + This informative section provides guidelines for supporting IRIs in + the same software components and operations that currently process + URIs: Software interfaces that handle URIs, software that allows + users to enter URIs, software that creates or generates URIs, + software that displays URIs, formats and protocols that transport + URIs, and software that interprets URIs. These may all require + modification before functioning properly with IRIs. The + considerations in this section also apply to URI references and IRI + references. + +7.1. URI/IRI Software Interfaces + + Software interfaces that handle URIs, such as URI-handling APIs and + protocols transferring URIs, need interfaces and protocol elements + that are designed to carry IRIs. + + In case the current handling in an API or protocol is based on + US-ASCII, UTF-8 is recommended as the character encoding for IRIs, as + it is compatible with US-ASCII, is in accordance with the + recommendations of [RFC2277], and makes converting to URIs easy. In + any case, the API or protocol definition must clearly define the + character encoding to be used. + + The transfer from URI-only to IRI-capable components requires no + mapping, although the conversion described in section 3.2 above may + be performed. It is preferable not to perform this inverse + conversion when there is a chance that this cannot be done correctly. + + + + + + +Duerst & Suignard Standards Track [Page 32] + +RFC 3987 Internationalized Resource Identifiers January 2005 + + +7.2. URI/IRI Entry + + Some components allow users to enter URIs into the system by typing + or dictation, for example. This software must be updated to allow + for IRI entry. + + A person viewing a visual representation of an IRI (as a sequence of + glyphs, in some order, in some visual display) or hearing an IRI will + use an entry method for characters in the user's language to input + the IRI. Depending on the script and the input method used, this may + be a more or less complicated process. + + The process of IRI entry must ensure, as much as possible, that the + restrictions defined in section 2.2 are met. This may be done by + choosing appropriate input methods or variants/settings thereof, by + appropriately converting the characters being input, by eliminating + characters that cannot be converted, and/or by issuing a warning or + error message to the user. + + As an example of variant settings, input method editors for East + Asian Languages usually allow the input of Latin letters and related + characters in full-width or half-width versions. For IRI input, the + input method editor should be set so that it produces half-width + Latin letters and punctuation and full-width Katakana. + + An input field primarily or solely used for the input of URIs/IRIs + may allow the user to view an IRI as it is mapped to a URI. Places + where the input of IRIs is frequent may provide the possibility for + viewing an IRI as mapped to a URI. This will help users when some of + the software they use does not yet accept IRIs. + + An IRI input component interfacing to components that handle URIs, + but not IRIs, must map the IRI to a URI before passing it to these + components. + + For the input of IRIs with right-to-left characters, please see + section 4.3. + +7.3. URI/IRI Transfer between Applications + + Many applications, particularly mail user agents, try to detect URIs + appearing in plain text. For this, they use some heuristics based on + URI syntax. They then allow the user to click on such URIs and + retrieve the corresponding resource in an appropriate (usually + scheme-dependent) application. + + + + + + +Duerst & Suignard Standards Track [Page 33] + +RFC 3987 Internationalized Resource Identifiers January 2005 + + + Such applications have to be upgraded to use the IRI syntax as a base + for heuristics. In particular, a non-ASCII character should not be + taken as the indication of the end of an IRI. Such applications also + have to make sure that they correctly convert the detected IRI from + the character encoding of the document or application where the IRI + appears to the character encoding used by the system-wide IRI + invocation mechanism, or to a URI (according to section 3.1) if the + system-wide invocation mechanism only accepts URIs. + + The clipboard is another frequently used way to transfer URIs and + IRIs from one application to another. On most platforms, the + clipboard is able to store and transfer text in many languages and + scripts. Correctly used, the clipboard transfers characters, not + bytes, which will do the right thing with IRIs. + +7.4. URI/IRI Generation + + Systems that offer resources through the Internet, where those + resources have logical names, sometimes automatically generate URIs + for the resources they offer. For example, some HTTP servers can + generate a directory listing for a file directory and then respond to + the generated URIs with the files. + + Many legacy character encodings are in use in various file systems. + Many currently deployed systems do not transform the local character + representation of the underlying system before generating URIs. + + For maximum interoperability, systems that generate resource + identifiers should make the appropriate transformations. For + example, if a file system contains a file named + "résumé.html", a server should expose this as + "r%C3%A9sum%C3%A9.html" in a URI, which allows use of + "résumé.html" in an IRI, even if locally the file name is + kept in a character encoding other than UTF-8. + + This recommendation particularly applies to HTTP servers. For FTP + servers, similar considerations apply; see [RFC2640]. + +7.5. URI/IRI Selection + + In some cases, resource owners and publishers have control over the + IRIs used to identify their resources. This control is mostly + executed by controlling the resource names, such as file names, + directly. + + + + + + + +Duerst & Suignard Standards Track [Page 34] + +RFC 3987 Internationalized Resource Identifiers January 2005 + + + In these cases, it is recommended to avoid choosing IRIs that are + easily confused. For example, for US-ASCII, the lower-case ell ("l") + is easily confused with the digit one ("1"), and the upper-case oh + ("O") is easily confused with the digit zero ("0"). Publishers + should avoid confusing users with "br0ken" or "1ame" identifiers. + + Outside the US-ASCII repertoire, there are many more opportunities + for confusion; a complete set of guidelines is too lengthy to include + here. As long as names are limited to characters from a single + script, native writers of a given script or language will know best + when ambiguities can appear, and how they can be avoided. What may + look ambiguous to a stranger may be completely obvious to the average + native user. On the other hand, in some cases, the UCS contains + variants for compatibility reasons; for example, for typographic + purposes. These should be avoided wherever possible. Although there + may be exceptions, newly created resource names should generally be + in NFKC [UTR15] (which means that they are also in NFC). + + As an example, the UCS contains the "fi" ligature at U+FB01 for + compatibility reasons. Wherever possible, IRIs should use the two + letters "f" and "i" rather than the "fi" ligature. An example where + the latter may be used is in the query part of an IRI for an explicit + search for a word written containing the "fi" ligature. + + In certain cases, there is a chance that characters from different + scripts look the same. The best known example is the similarity of + the Latin "A", the Greek "Alpha", and the Cyrillic "A". To avoid + such cases, only IRIs should be created where all the characters in a + single component are used together in a given language. This usually + means that all of these characters will be from the same script, but + there are languages that mix characters from different scripts (such + as Japanese). This is similar to the heuristics used to distinguish + between letters and numbers in the examples above. Also, for Latin, + Greek, and Cyrillic, using lowercase letters results in fewer + ambiguities than using uppercase letters would. + +7.6. Display of URIs/IRIs + + In situations where the rendering software is not expected to display + non-ASCII parts of the IRI correctly using the available layout and + font resources, these parts should be percent-encoded before being + displayed. + + For display of Bidi IRIs, please see section 4.1. + + + + + + + +Duerst & Suignard Standards Track [Page 35] + +RFC 3987 Internationalized Resource Identifiers January 2005 + + +7.7. Interpretation of URIs and IRIs + + Software that interprets IRIs as the names of local resources should + accept IRIs in multiple forms and convert and match them with the + appropriate local resource names. + + First, multiple representations include both IRIs in the native + character encoding of the protocol and also their URI counterparts. + + Second, it may include URIs constructed based on character encodings + other than UTF-8. These URIs may be produced by user agents that do + not conform to this specification and that use legacy character + encodings to convert non-ASCII characters to URIs. Whether this is + necessary, and what character encodings to cover, depends on a number + of factors, such as the legacy character encodings used locally and + the distribution of various versions of user agents. For example, + software for Japanese may accept URIs in Shift_JIS and/or EUC-JP in + addition to UTF-8. + + Third, it may include additional mappings to be more user-friendly + and robust against transmission errors. These would be similar to + how some servers currently treat URIs as case insensitive or perform + additional matching to account for spelling errors. For characters + beyond the US-ASCII repertoire, this may, for example, include + ignoring the accents on received IRIs or resource names. Please note + that such mappings, including case mappings, are language dependent. + + It can be difficult to identify a resource unambiguously if too many + mappings are taken into consideration. However, percent-encoded and + not percent-encoded parts of IRIs can always be clearly + distinguished. Also, the regularity of UTF-8 (see [Duerst97]) makes + the potential for collisions lower than it may seem at first. + +7.8. Upgrading Strategy + + Where this recommendation places further constraints on software for + which many instances are already deployed, it is important to + introduce upgrades carefully and to be aware of the various + interdependencies. + + If IRIs cannot be interpreted correctly, they should not be created, + generated, or transported. This suggests that upgrading URI + interpreting software to accept IRIs should have highest priority. + + On the other hand, a single IRI is interpreted only by a single or + very few interpreters that are known in advance, although it may be + entered and transported very widely. + + + + +Duerst & Suignard Standards Track [Page 36] + +RFC 3987 Internationalized Resource Identifiers January 2005 + + + Therefore, IRIs benefit most from a broad upgrade of software to be + able to enter and transport IRIs. However, before an individual IRI + is published, care should be taken to upgrade the corresponding + interpreting software in order to cover the forms expected to be + received by various versions of entry and transport software. + + The upgrade of generating software to generate IRIs instead of using + a local character encoding should happen only after the service is + upgraded to accept IRIs. Similarly, IRIs should only be generated + when the service accepts IRIs and the intervening infrastructure and + protocol is known to transport them safely. + + Software converting from URIs to IRIs for display should be upgraded + only after upgraded entry software has been widely deployed to the + population that will see the displayed result. + + Where there is a free choice of character encodings, it is often + possible to reduce the effort and dependencies for upgrading to IRIs + by using UTF-8 rather than another encoding. For example, when a new + file-based Web server is set up, using UTF-8 as the character + encoding for file names will make the transition to IRIs easier. + Likewise, when a new Web form is set up using UTF-8 as the character + encoding of the form page, the returned query URIs will use UTF-8 as + the character encoding (unless the user, for whatever reason, changes + the character encoding) and will therefore be compatible with IRIs. + + These recommendations, when taken together, will allow for the + extension from URIs to IRIs in order to handle characters other than + US-ASCII while minimizing interoperability problems. For + considerations regarding the upgrade of URI scheme definitions, see + section 6.4. + +8. Security Considerations + + The security considerations discussed in [RFC3986] also apply to + IRIs. In addition, the following issues require particular care for + IRIs. + + Incorrect encoding or decoding can lead to security problems. In + particular, some UTF-8 decoders do not check against overlong byte + sequences. As an example, a "/" is encoded with the byte 0x2F both + in UTF-8 and in US-ASCII, but some UTF-8 decoders also wrongly + interpret the sequence 0xC0 0xAF as a "/". A sequence such as + + + + + + + + +Duerst & Suignard Standards Track [Page 37] + +RFC 3987 Internationalized Resource Identifiers January 2005 + + + "%C0%AF.." may pass some security tests and then be interpreted as + "/.." in a path if UTF-8 decoders are fault-tolerant, if conversion + and checking are not done in the right order, and/or if reserved + characters and unreserved characters are not clearly distinguished. + + There are various ways in which "spoofing" can occur with IRIs. + "Spoofing" means that somebody may add a resource name that looks the + same or similar to the user, but that points to a different resource. + The added resource may pretend to be the real resource by looking + very similar but may contain all kinds of changes that may be + difficult to spot and that can cause all kinds of problems. Most + spoofing possibilities for IRIs are extensions of those for URIs. + + Spoofing can occur for various reasons. First, a user's + normalization expectations or actual normalization when entering an + IRI or transcoding an IRI from a legacy character encoding do not + match the normalization used on the server side. Conceptually, this + is no different from the problems surrounding the use of + case-insensitive web servers. For example, a popular web page with a + mixed-case name ("http://big.example.com/PopularPage.html") might be + "spoofed" by someone who is able to create + "http://big.example.com/popularpage.html". However, the use of + unnormalized character sequences, and of additional mappings for user + convenience, may increase the chance for spoofing. Protocols and + servers that allow the creation of resources with names that are not + normalized are particularly vulnerable to such attacks. This is an + inherent security problem of the relevant protocol, server, or + resource and is not specific to IRIs, but it is mentioned here for + completeness. + + Spoofing can occur in various IRI components, such as the domain name + part or a path part. For considerations specific to the domain name + part, see [RFC3491]. For the path part, administrators of sites that + allow independent users to create resources in the same sub area may + have to be careful to check for spoofing. + + Spoofing can occur because in the UCS many characters look very + similar. Details are discussed in Section 7.5. Again, this is very + similar to spoofing possibilities on US-ASCII, e.g., using "br0ken" + or "1ame" URIs. + + Spoofing can occur when URIs with percent-encodings based on various + character encodings are accepted to deal with older user agents. In + some cases, particularly for Latin-based resource names, this is + usually easy to detect because UTF-8-encoded names, when interpreted + and viewed as legacy character encodings, produce mostly garbage. + + + + + +Duerst & Suignard Standards Track [Page 38] + +RFC 3987 Internationalized Resource Identifiers January 2005 + + + When concurrently used character encodings have a similar structure + but there are no characters that have exactly the same encoding, + detection is more difficult. + + Spoofing can occur with bidirectional IRIs, if the restrictions in + section 4.2 are not followed. The same visual representation may be + interpreted as different logical representations, and vice versa. It + is also very important that a correct Unicode bidirectional + implementation be used. + +9. Acknowledgements + + We would like to thank Larry Masinter for his work as coauthor of + many earlier versions of this document (draft-masinter-url-i18n-xx). + + The discussion on the issue addressed here started a long time ago. + There was a thread in the HTML working group in August 1995 (under + the topic of "Globalizing URIs") and in the www-international mailing + list in July 1996 (under the topic of "Internationalization and + URLs"), and there were ad-hoc meetings at the Unicode conferences in + September 1995 and September 1997. + + Many thanks go to Francois Yergeau, Matitiahu Allouche, Roy Fielding, + Tim Berners-Lee, Mark Davis, M.T. Carrasco Benitez, James Clark, Tim + Bray, Chris Wendt, Yaron Goland, Andrea Vine, Misha Wolf, Leslie + Daigle, Ted Hardie, Bill Fenner, Margaret Wasserman, Russ Housley, + Makoto MURATA, Steven Atkin, Ryan Stansifer, Tex Texin, Graham Klyne, + Bjoern Hoehrmann, Chris Lilley, Ian Jacobs, Adam Costello, Dan + Oscarson, Elliotte Rusty Harold, Mike J. Brown, Roy Badami, Jonathan + Rosenne, Asmus Freytag, Simon Josefsson, Carlos Viegas Damasio, Chris + Haynes, Walter Underwood, and many others for help with understanding + the issues and possible solutions, and with getting the details + right. + + This document is a product of the Internationalization Working Group + (I18N WG) of the World Wide Web Consortium (W3C). Thanks to the + members of the W3C I18N Working Group and Interest Group for their + contributions and their work on [CharMod]. Thanks also go to the + members of many other W3C Working Groups for adopting IRIs, and to + the members of the Montreal IAB Workshop on Internationalization and + Localization for their review. + + + + + + + + + + +Duerst & Suignard Standards Track [Page 39] + +RFC 3987 Internationalized Resource Identifiers January 2005 + + +10. References + +10.1. Normative References + + [ASCII] American National Standards Institute, "Coded + Character Set -- 7-bit American Standard Code for + Information Interchange", ANSI X3.4, 1986. + + [ISO10646] International Organization for Standardization, + "ISO/IEC 10646:2003: Information Technology - + Universal Multiple-Octet Coded Character Set (UCS)", + ISO Standard 10646, December 2003. + + [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate + Requirement Levels", BCP 14, RFC 2119, March 1997. + + [RFC2234] Crocker, D. and P. Overell, "Augmented BNF for Syntax + Specifications: ABNF", RFC 2234, November 1997. + + [RFC3490] Faltstrom, P., Hoffman, P., and A. Costello, + "Internationalizing Domain Names in Applications + (IDNA)", RFC 3490, March 2003. + + [RFC3491] Hoffman, P. and M. Blanchet, "Nameprep: A Stringprep + Profile for Internationalized Domain Names (IDN)", RFC + 3491, March 2003. + + [RFC3629] Yergeau, F., "UTF-8, a transformation format of ISO + 10646", STD 63, RFC 3629, November 2003. + + [RFC3986] Berners-Lee, T., Fielding, R., and L. Masinter, + "Uniform Resource Identifier (URI): Generic Syntax", + STD 66, RFC 3986, January 2005. + + [UNI9] Davis, M., "The Bidirectional Algorithm", Unicode + Standard Annex #9, March 2004, + . + + [UNIV4] The Unicode Consortium, "The Unicode Standard, Version + 4.0.1, defined by: The Unicode Standard, Version 4.0 + (Reading, MA, Addison-Wesley, 2003. ISBN + 0-321-18578-1), as amended by Unicode 4.0.1 + (http://www.unicode.org/versions/Unicode4.0.1/)", + March 2004. + + + + + + + +Duerst & Suignard Standards Track [Page 40] + +RFC 3987 Internationalized Resource Identifiers January 2005 + + + [UTR15] Davis, M. and M. Duerst, "Unicode Normalization + Forms", Unicode Standard Annex #15, April 2003, + . + +10.2. Informative References + + [BidiEx] "Examples of bidirectional IRIs", + . + + [CharMod] Duerst, M., Yergeau, F., Ishida, R., Wolf, M., and T. + Texin, "Character Model for the World Wide Web: + Resource Identifiers", World Wide Web Consortium + Candidate Recommendation, November 2004, + . + + [Duerst97] Duerst, M., "The Properties and Promises of UTF-8", + Proc. 11th International Unicode Conference, San Jose + , September 1997, + . + + [Gettys] Gettys, J., "URI Model Consequences", + . + + [HTML4] Raggett, D., Le Hors, A., and I. Jacobs, "HTML 4.01 + Specification", World Wide Web Consortium + Recommendation, December 1999, + . + + [RFC2045] Freed, N. and N. Borenstein, "Multipurpose Internet + Mail Extensions (MIME) Part One: Format of Internet + Message Bodies", RFC 2045, November 1996. + + [RFC2130] Weider, C., Preston, C., Simonsen, K., Alvestrand, H., + Atkinson, R., Crispin, M., and P. Svanberg, "The + Report of the IAB Character Set Workshop held 29 + February - 1 March, 1996", RFC 2130, April 1997. + + [RFC2141] Moats, R., "URN Syntax", RFC 2141, May 1997. + + [RFC2192] Newman, C., "IMAP URL Scheme", RFC 2192, September + 1997. + + [RFC2277] Alvestrand, H., "IETF Policy on Character Sets and + Languages", BCP 18, RFC 2277, January 1998. + + + +Duerst & Suignard Standards Track [Page 41] + +RFC 3987 Internationalized Resource Identifiers January 2005 + + + [RFC2368] Hoffman, P., Masinter, L., and J. Zawinski, "The + mailto URL scheme", RFC 2368, July 1998. + + [RFC2384] Gellens, R., "POP URL Scheme", RFC 2384, August 1998. + + [RFC2396] Berners-Lee, T., Fielding, R., and L. Masinter, + "Uniform Resource Identifiers (URI): Generic Syntax", + RFC 2396, August 1998. + + [RFC2397] Masinter, L., "The "data" URL scheme", RFC 2397, + August 1998. + + [RFC2616] Fielding, R., Gettys, J., Mogul, J., Frystyk, H., + Masinter, L., Leach, P., and T. Berners-Lee, + "Hypertext Transfer Protocol -- HTTP/1.1", RFC 2616, + June 1999. + + [RFC2640] Curtin, B., "Internationalization of the File Transfer + Protocol", RFC 2640, July 1999. + + [RFC2718] Masinter, L., Alvestrand, H., Zigmond, D., and R. + Petke, "Guidelines for new URL Schemes", RFC 2718, + November 1999. + + [UNIXML] Duerst, M. and A. Freytag, "Unicode in XML and other + Markup Languages", Unicode Technical Report #20, World + Wide Web Consortium Note, June 2003, + . + + [XLink] DeRose, S., Maler, E., and D. Orchard, "XML Linking + Language (XLink) Version 1.0", World Wide Web + Consortium Recommendation, June 2001, + . + + [XML1] Bray, T., Paoli, J., Sperberg-McQueen, C., Maler, E., + and F. Yergeau, "Extensible Markup Language (XML) 1.0 + (Third Edition)", World Wide Web Consortium + Recommendation, February 2004, + . + + [XMLNamespace] Bray, T., Hollander, D., and A. Layman, "Namespaces in + XML", World Wide Web Consortium Recommendation, + January 1999, . + + [XMLSchema] Biron, P. and A. Malhotra, "XML Schema Part 2: + Datatypes", World Wide Web Consortium Recommendation, + May 2001, . + + + + +Duerst & Suignard Standards Track [Page 42] + +RFC 3987 Internationalized Resource Identifiers January 2005 + + + [XPointer] Grosso, P., Maler, E., Marsh, J. and N. Walsh, + "XPointer Framework", World Wide Web Consortium + Recommendation, March 2003, + . + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +Duerst & Suignard Standards Track [Page 43] + +RFC 3987 Internationalized Resource Identifiers January 2005 + + +Appendix A. Design Alternatives + + This section shortly summarizes major design alternatives and the + reasons for why they were not chosen. + +Appendix A.1. New Scheme(s) + + Introducing new schemes (for example, httpi:, ftpi:,...) or a new + metascheme (e.g., i:, leading to URI/IRI prefixes such as i:http:, + i:ftp:,...) was proposed to make IRI-to-URI conversion scheme + dependent or to distinguish between percent-encodings resulting from + IRI-to-URI conversion and percent-encodings from legacy character + encodings. + + New schemes are not needed to distinguish URIs from true IRIs (i.e., + IRIs that contain non-ASCII characters). The benefit of being able + to detect the origin of percent-encodings is marginal, as UTF-8 can + be detected with very high reliability. Deploying new schemes is + extremely hard, so not requiring new schemes for IRIs makes + deployment of IRIs vastly easier. Making conversion scheme dependent + is highly inadvisable and would be encouraged by separate schemes for + IRIs. Using a uniform convention for conversion from IRIs to URIs + makes IRI implementation orthogonal to the introduction of actual new + schemes. + +Appendix A.2. Character Encodings Other Than UTF-8 + + At an early stage, UTF-7 was considered as an alternative to UTF-8 + when IRIs are converted to URIs. UTF-7 would not have needed + percent-encoding and in most cases would have been shorter than + percent-encoded UTF-8. + + Using UTF-8 avoids a double layering and overloading of the use of + the "+" character. UTF-8 is fully compatible with US-ASCII and has + therefore been recommended by the IETF, and is being used widely. + + UTF-7 has never been used much and is now clearly being discouraged. + Requiring implementations to convert from UTF-8 to UTF-7 and back + would be an additional implementation burden. + +Appendix A.3. New Encoding Convention + + Instead of using the existing percent-encoding convention of URIs, + which is based on octets, the idea was to create a new encoding + convention; for example, to use "%u" to introduce UCS code points. + + + + + + +Duerst & Suignard Standards Track [Page 44] + +RFC 3987 Internationalized Resource Identifiers January 2005 + + + Using the existing octet-based percent-encoding mechanism does not + need an upgrade of the URI syntax and does not need corresponding + server upgrades. + +Appendix A.4. Indicating Character Encodings in the URI/IRI + + Some proposals suggested indicating the character encodings used in + an URI or IRI with some new syntactic convention in the URI itself, + similar to the "charset" parameter for e-mails and Web pages. As an + example, the label in square brackets in + "http://www.example.org/ros[iso-8859-1]é"; indicated that the + following "é"; had to be interpreted as iso-8859-1. + + If UTF-8 is used exclusively, an upgrade to the URI syntax is not + needed. It avoids potentially multiple labels that have to be copied + correctly in all cases, even on the side of a bus or on a napkin, + leading to usability problems (and being prohibitively annoying). + Exclusively using UTF-8 also reduces transcoding errors and + confusion. + +Authors' Addresses + + Martin Duerst (Note: Please write "Duerst" with u-umlaut wherever + possible, for example as "Dürst" in XML and + HTML.) + World Wide Web Consortium + 5322 Endo + Fujisawa, Kanagawa 252-8520 + Japan + + Phone: +81 466 49 1170 + Fax: +81 466 49 1171 + EMail: duerst@w3.org + URI: http://www.w3.org/People/D%C3%BCrst/ + (Note: This is the percent-encoded form of an IRI.) + + + Michel Suignard + Microsoft Corporation + One Microsoft Way + Redmond, WA 98052 + U.S.A. + + Phone: +1 425 882-8080 + EMail: michelsu@microsoft.com + URI: http://www.suignard.com + + + + + +Duerst & Suignard Standards Track [Page 45] + +RFC 3987 Internationalized Resource Identifiers January 2005 + + +Full Copyright Statement + + Copyright (C) The Internet Society (2005). + + This document is subject to the rights, licenses and restrictions + contained in BCP 78, and except as set forth therein, the authors + retain all their rights. + + This document and the information contained herein are provided on an + "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS + OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET + ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, + INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE + INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED + WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. + +Intellectual Property + + The IETF takes no position regarding the validity or scope of any + Intellectual Property Rights or other rights that might be claimed to + pertain to the implementation or use of the technology described in + this document or the extent to which any license under such rights + might or might not be available; nor does it represent that it has + made any independent effort to identify any such rights. Information + on the IETF's procedures with respect to rights in IETF Documents can + be found in BCP 78 and BCP 79. + + Copies of IPR disclosures made to the IETF Secretariat and any + assurances of licenses to be made available, or the result of an + attempt made to obtain a general license or permission for the use of + such proprietary rights by implementers or users of this + specification can be obtained from the IETF on-line IPR repository at + http://www.ietf.org/ipr. + + The IETF invites any interested party to bring to its attention any + copyrights, patents or patent applications, or other proprietary + rights that may cover technology that may be required to implement + this standard. Please address the information to the IETF at ietf- + ipr@ietf.org. + + +Acknowledgement + + Funding for the RFC Editor function is currently provided by the + Internet Society. + + + + + + +Duerst & Suignard Standards Track [Page 46] + diff --git a/docs/rfc5987.txt b/docs/rfc5987.txt new file mode 100644 index 0000000..37cac39 --- /dev/null +++ b/docs/rfc5987.txt @@ -0,0 +1,563 @@ + + + + + + +Internet Engineering Task Force (IETF) J. Reschke +Request for Comments: 5987 greenbytes +Category: Standards Track August 2010 +ISSN: 2070-1721 + + + Character Set and Language Encoding for + Hypertext Transfer Protocol (HTTP) Header Field Parameters + +Abstract + + By default, message header field parameters in Hypertext Transfer + Protocol (HTTP) messages cannot carry characters outside the ISO- + 8859-1 character set. RFC 2231 defines an encoding mechanism for use + in Multipurpose Internet Mail Extensions (MIME) headers. This + document specifies an encoding suitable for use in HTTP header fields + that is compatible with a profile of the encoding defined in RFC + 2231. + +Status of This Memo + + This is an Internet Standards Track document. + + This document is a product of the Internet Engineering Task Force + (IETF). It represents the consensus of the IETF community. It has + received public review and has been approved for publication by the + Internet Engineering Steering Group (IESG). Further information on + Internet Standards is available in Section 2 of RFC 5741. + + Information about the current status of this document, any errata, + and how to provide feedback on it may be obtained at + http://www.rfc-editor.org/info/rfc5987. + +Copyright Notice + + Copyright (c) 2010 IETF Trust and the persons identified as the + document authors. All rights reserved. + + This document is subject to BCP 78 and the IETF Trust's Legal + Provisions Relating to IETF Documents + (http://trustee.ietf.org/license-info) in effect on the date of + publication of this document. Please review these documents + carefully, as they describe your rights and restrictions with respect + to this document. Code Components extracted from this document must + include Simplified BSD License text as described in Section 4.e of + the Trust Legal Provisions and are provided without warranty as + described in the Simplified BSD License. + + + + +Reschke Standards Track [Page 1] + +RFC 5987 Charset/Language Encoding in HTTP August 2010 + + +Table of Contents + + 1. Introduction ....................................................2 + 2. Notational Conventions ..........................................2 + 3. Comparison to RFC 2231 and Definition of the Encoding ...........3 + 3.1. Parameter Continuations ....................................3 + 3.2. Parameter Value Character Set and Language Information .....3 + 3.2.1. Definition ..........................................3 + 3.2.2. Examples ............................................6 + 3.3. Language Specification in Encoded Words ....................6 + 4. Guidelines for Usage in HTTP Header Field Definitions ...........7 + 4.1. When to Use the Extension ..................................7 + 4.2. Error Handling .............................................7 + 5. Security Considerations .........................................8 + 6. Acknowledgements ................................................8 + 7. References ......................................................8 + 7.1. Normative References .......................................8 + 7.2. Informative References .....................................9 + +1. Introduction + + By default, message header field parameters in HTTP ([RFC2616]) + messages cannot carry characters outside the ISO-8859-1 character set + ([ISO-8859-1]). RFC 2231 ([RFC2231]) defines an encoding mechanism + for use in MIME headers. This document specifies an encoding + suitable for use in HTTP header fields that is compatible with a + profile of the encoding defined in RFC 2231. + + Note: in the remainder of this document, RFC 2231 is only + referenced for the purpose of explaining the choice of features + that were adopted; they are therefore purely informative. + + Note: this encoding does not apply to message payloads transmitted + over HTTP, such as when using the media type "multipart/form-data" + ([RFC2388]). + +2. Notational Conventions + + The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", + "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this + document are to be interpreted as described in [RFC2119]. + + This specification uses the ABNF (Augmented Backus-Naur Form) + notation defined in [RFC5234]. The following core rules are included + by reference, as defined in [RFC5234], Appendix B.1: ALPHA (letters), + DIGIT (decimal 0-9), HEXDIG (hexadecimal 0-9/A-F/a-f), and LWSP + (linear whitespace). + + + + +Reschke Standards Track [Page 2] + +RFC 5987 Charset/Language Encoding in HTTP August 2010 + + + Note that this specification uses the term "character set" for + consistency with other IETF specifications such as RFC 2277 (see + [RFC2277], Section 3). A more accurate term would be "character + encoding" (a mapping of code points to octet sequences). + +3. Comparison to RFC 2231 and Definition of the Encoding + + RFC 2231 defines several extensions to MIME. The sections below + discuss if and how they apply to HTTP header fields. + + In short: + + o Parameter Continuations aren't needed (Section 3.1), + + o Character Set and Language Information are useful, therefore a + simple subset is specified (Section 3.2), and + + o Language Specifications in Encoded Words aren't needed + (Section 3.3). + +3.1. Parameter Continuations + + Section 3 of [RFC2231] defines a mechanism that deals with the length + limitations that apply to MIME headers. These limitations do not + apply to HTTP ([RFC2616], Section 19.4.7). + + Thus, parameter continuations are not part of the encoding defined by + this specification. + +3.2. Parameter Value Character Set and Language Information + + Section 4 of [RFC2231] specifies how to embed language information + into parameter values, and also how to encode non-ASCII characters, + dealing with restrictions both in MIME and HTTP header parameters. + + However, RFC 2231 does not specify a mandatory-to-implement character + set, making it hard for senders to decide which character set to use. + Thus, recipients implementing this specification MUST support the + character sets "ISO-8859-1" [ISO-8859-1] and "UTF-8" [RFC3629]. + + Furthermore, RFC 2231 allows the character set information to be left + out. The encoding defined by this specification does not allow that. + +3.2.1. Definition + + The syntax for parameters is defined in Section 3.6 of [RFC2616] + (with RFC 2616 implied LWS translated to RFC 5234 LWSP): + + + + +Reschke Standards Track [Page 3] + +RFC 5987 Charset/Language Encoding in HTTP August 2010 + + + parameter = attribute LWSP "=" LWSP value + + attribute = token + value = token / quoted-string + + quoted-string = + token = + + In order to include character set and language information, this + specification modifies the RFC 2616 grammar to be: + + parameter = reg-parameter / ext-parameter + + reg-parameter = parmname LWSP "=" LWSP value + + ext-parameter = parmname "*" LWSP "=" LWSP ext-value + + parmname = 1*attr-char + + ext-value = charset "'" [ language ] "'" value-chars + ; like RFC 2231's + ; (see [RFC2231], Section 7) + + charset = "UTF-8" / "ISO-8859-1" / mime-charset + + mime-charset = 1*mime-charsetc + mime-charsetc = ALPHA / DIGIT + / "!" / "#" / "$" / "%" / "&" + / "+" / "-" / "^" / "_" / "`" + / "{" / "}" / "~" + ; as in Section 2.3 of [RFC2978] + ; except that the single quote is not included + ; SHOULD be registered in the IANA charset registry + + language = + + value-chars = *( pct-encoded / attr-char ) + + pct-encoded = "%" HEXDIG HEXDIG + ; see [RFC3986], Section 2.1 + + attr-char = ALPHA / DIGIT + / "!" / "#" / "$" / "&" / "+" / "-" / "." + / "^" / "_" / "`" / "|" / "~" + ; token except ( "*" / "'" / "%" ) + + + + + + +Reschke Standards Track [Page 4] + +RFC 5987 Charset/Language Encoding in HTTP August 2010 + + + Thus, a parameter is either a regular parameter (reg-parameter), as + previously defined in Section 3.6 of [RFC2616], or an extended + parameter (ext-parameter). + + Extended parameters are those where the left-hand side of the + assignment ends with an asterisk character. + + The value part of an extended parameter (ext-value) is a token that + consists of three parts: the REQUIRED character set name (charset), + the OPTIONAL language information (language), and a character + sequence representing the actual value (value-chars), separated by + single quote characters. Note that both character set names and + language tags are restricted to the US-ASCII character set, and are + matched case-insensitively (see [RFC2978], Section 2.3 and [RFC5646], + Section 2.1.1). + + Inside the value part, characters not contained in attr-char are + encoded into an octet sequence using the specified character set. + That octet sequence is then percent-encoded as specified in Section + 2.1 of [RFC3986]. + + Producers MUST use either the "UTF-8" ([RFC3629]) or the "ISO-8859-1" + ([ISO-8859-1]) character set. Extension character sets (mime- + charset) are reserved for future use. + + Note: recipients should be prepared to handle encoding errors, + such as malformed or incomplete percent escape sequences, or non- + decodable octet sequences, in a robust manner. This specification + does not mandate any specific behavior, for instance, the + following strategies are all acceptable: + + * ignoring the parameter, + + * stripping a non-decodable octet sequence, + + * substituting a non-decodable octet sequence by a replacement + character, such as the Unicode character U+FFFD (Replacement + Character). + + Note: the RFC 2616 token production ([RFC2616], Section 2.2) + differs from the production used in RFC 2231 (imported from + Section 5.1 of [RFC2045]) in that curly braces ("{" and "}") are + excluded. Thus, these two characters are excluded from the attr- + char production as well. + + + + + + + +Reschke Standards Track [Page 5] + +RFC 5987 Charset/Language Encoding in HTTP August 2010 + + + Note: the ABNF defined here differs from the one in + Section 2.3 of [RFC2978] in that it does not allow the single + quote character (see also RFC Errata ID 1912 [Err1912]). In + practice, no character set names using that character have been + registered at the time of this writing. + +3.2.2. Examples + + Non-extended notation, using "token": + + foo: bar; title=Economy + + Non-extended notation, using "quoted-string": + + foo: bar; title="US-$ rates" + + Extended notation, using the Unicode character U+00A3 (POUND SIGN): + + foo: bar; title*=iso-8859-1'en'%A3%20rates + + Note: the Unicode pound sign character U+00A3 was encoded into the + single octet A3 using the ISO-8859-1 character encoding, then + percent-encoded. Also, note that the space character was encoded as + %20, as it is not contained in attr-char. + + Extended notation, using the Unicode characters U+00A3 (POUND SIGN) + and U+20AC (EURO SIGN): + + foo: bar; title*=UTF-8''%c2%a3%20and%20%e2%82%ac%20rates + + Note: the Unicode pound sign character U+00A3 was encoded into the + octet sequence C2 A3 using the UTF-8 character encoding, then + percent-encoded. Likewise, the Unicode euro sign character U+20AC + was encoded into the octet sequence E2 82 AC, then percent-encoded. + Also note that HEXDIG allows both lowercase and uppercase characters, + so recipients must understand both, and that the language information + is optional, while the character set is not. + +3.3. Language Specification in Encoded Words + + Section 5 of [RFC2231] extends the encoding defined in [RFC2047] to + also support language specification in encoded words. Although the + HTTP/1.1 specification does refer to RFC 2047 ([RFC2616], Section + 2.2), it's not clear to which header field exactly it applies, and + whether it is implemented in practice (see + for details). + + Thus, this specification does not include this feature. + + + +Reschke Standards Track [Page 6] + +RFC 5987 Charset/Language Encoding in HTTP August 2010 + + +4. Guidelines for Usage in HTTP Header Field Definitions + + Specifications of HTTP header fields that use the extensions defined + in Section 3.2 ought to clearly state that. A simple way to achieve + this is to normatively reference this specification, and to include + the ext-value production into the ABNF for that header field. + + For instance: + + foo-header = "foo" LWSP ":" LWSP token ";" LWSP title-param + title-param = "title" LWSP "=" LWSP value + / "title*" LWSP "=" LWSP ext-value + ext-value = + + Note: The Parameter Value Continuation feature defined in Section + 3 of [RFC2231] makes it impossible to have multiple instances of + extended parameters with identical parmname components, as the + processing of continuations would become ambiguous. Thus, + specifications using this extension are advised to disallow this + case for compatibility with RFC 2231. + +4.1. When to Use the Extension + + Section 4.2 of [RFC2277] requires that protocol elements containing + human-readable text are able to carry language information. Thus, + the ext-value production ought to be always used when the parameter + value is of textual nature and its language is known. + + Furthermore, the extension ought to also be used whenever the + parameter value needs to carry characters not present in the US-ASCII + ([USASCII]) character set (note that it would be unacceptable to + define a new parameter that would be restricted to a subset of the + Unicode character set). + +4.2. Error Handling + + Header field specifications need to define whether multiple instances + of parameters with identical parmname components are allowed, and how + they should be processed. This specification suggests that a + parameter using the extended syntax takes precedence. This would + allow producers to use both formats without breaking recipients that + do not understand the extended syntax yet. + + Example: + + foo: bar; title="EURO exchange rates"; + title*=utf-8''%e2%82%ac%20exchange%20rates + + + + +Reschke Standards Track [Page 7] + +RFC 5987 Charset/Language Encoding in HTTP August 2010 + + + In this case, the sender provides an ASCII version of the title for + legacy recipients, but also includes an internationalized version for + recipients understanding this specification -- the latter obviously + ought to prefer the new syntax over the old one. + + Note: at the time of this writing, many implementations failed to + ignore the form they do not understand, or prioritize the ASCII + form although the extended syntax was present. + +5. Security Considerations + + The format described in this document makes it possible to transport + non-ASCII characters, and thus enables character "spoofing" + scenarios, in which a displayed value appears to be something other + than it is. + + Furthermore, there are known attack scenarios relating to decoding + UTF-8. + + See Section 10 of [RFC3629] for more information on both topics. + + In addition, the extension specified in this document makes it + possible to transport multiple language variants for a single + parameter, and such use might allow spoofing attacks, where different + language versions of the same parameter are not equivalent. Whether + this attack is useful as an attack depends on the parameter + specified. + +6. Acknowledgements + + Thanks to Martin Duerst and Frank Ellermann for help figuring out + ABNF details, to Graham Klyne and Alexey Melnikov for general review, + to Chris Newman for pointing out an RFC 2231 incompatibility, and to + Benjamin Carlyle and Roar Lauritzsen for implementer's feedback. + +7. References + +7.1. Normative References + + [ISO-8859-1] International Organization for Standardization, + "Information technology -- 8-bit single-byte coded + graphic character sets -- Part 1: Latin alphabet No. + 1", ISO/IEC 8859-1:1998, 1998. + + [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate + Requirement Levels", BCP 14, RFC 2119, March 1997. + + + + + +Reschke Standards Track [Page 8] + +RFC 5987 Charset/Language Encoding in HTTP August 2010 + + + [RFC2616] Fielding, R., Gettys, J., Mogul, J., Frystyk, H., + Masinter, L., Leach, P., and T. Berners-Lee, "Hypertext + Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999. + + [RFC2978] Freed, N. and J. Postel, "IANA Charset Registration + Procedures", BCP 19, RFC 2978, October 2000. + + [RFC3629] Yergeau, F., "UTF-8, a transformation format of ISO + 10646", RFC 3629, STD 63, November 2003. + + [RFC3986] Berners-Lee, T., Fielding, R., and L. Masinter, + "Uniform Resource Identifier (URI): Generic Syntax", + RFC 3986, STD 66, January 2005. + + [RFC5234] Crocker, D., Ed. and P. Overell, "Augmented BNF for + Syntax Specifications: ABNF", STD 68, RFC 5234, + January 2008. + + [RFC5646] Phillips, A., Ed. and M. Davis, Ed., "Tags for + Identifying Languages", BCP 47, RFC 5646, + September 2009. + + [USASCII] American National Standards Institute, "Coded Character + Set -- 7-bit American Standard Code for Information + Interchange", ANSI X3.4, 1986. + +7.2. Informative References + + [Err1912] RFC Errata, Errata ID 1912, RFC 2978, + . + + [RFC2045] Freed, N. and N. Borenstein, "Multipurpose Internet + Mail Extensions (MIME) Part One: Format of Internet + Message Bodies", RFC 2045, November 1996. + + [RFC2047] Moore, K., "MIME (Multipurpose Internet Mail + Extensions) Part Three: Message Header Extensions for + Non-ASCII Text", RFC 2047, November 1996. + + [RFC2231] Freed, N. and K. Moore, "MIME Parameter Value and + Encoded Word Extensions: Character Sets, Languages, and + Continuations", RFC 2231, November 1997. + + [RFC2277] Alvestrand, H., "IETF Policy on Character Sets and + Languages", BCP 18, RFC 2277, January 1998. + + [RFC2388] Masinter, L., "Returning Values from Forms: multipart/ + form-data", RFC 2388, August 1998. + + + +Reschke Standards Track [Page 9] + +RFC 5987 Charset/Language Encoding in HTTP August 2010 + + +Author's Address + + Julian F. Reschke + greenbytes GmbH + Hafenweg 16 + Muenster, NW 48155 + Germany + + EMail: julian.reschke@greenbytes.de + URI: http://greenbytes.de/tech/webdav/ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +Reschke Standards Track [Page 10] + diff --git a/docs/rfc5988.txt b/docs/rfc5988.txt new file mode 100644 index 0000000..6f60ade --- /dev/null +++ b/docs/rfc5988.txt @@ -0,0 +1,1291 @@ + + + + + + +Internet Engineering Task Force (IETF) M. Nottingham +Request for Comments: 5988 October 2010 +Updates: 4287 +Category: Standards Track +ISSN: 2070-1721 + + + Web Linking + +Abstract + + This document specifies relation types for Web links, and defines a + registry for them. It also defines the use of such links in HTTP + headers with the Link header field. + +Status of This Memo + + This is an Internet Standards Track document. + + This document is a product of the Internet Engineering Task Force + (IETF). It represents the consensus of the IETF community. It has + received public review and has been approved for publication by the + Internet Engineering Steering Group (IESG). Further information on + Internet Standards is available in Section 2 of RFC 5741. + + Information about the current status of this document, any errata, + and how to provide feedback on it may be obtained at + http://www.rfc-editor.org/info/rfc5988. + +Copyright Notice + + Copyright (c) 2010 IETF Trust and the persons identified as the + document authors. All rights reserved. + + This document is subject to BCP 78 and the IETF Trust's Legal + Provisions Relating to IETF Documents + (http://trustee.ietf.org/license-info) in effect on the date of + publication of this document. Please review these documents + carefully, as they describe your rights and restrictions with respect + to this document. Code Components extracted from this document must + include Simplified BSD License text as described in Section 4.e of + the Trust Legal Provisions and are provided without warranty as + described in the Simplified BSD License. + + + + + + + + +Nottingham Standards Track [Page 1] + +RFC 5988 Web Linking October 2010 + + + This document may contain material from IETF Documents or IETF + Contributions published or made publicly available before November + 10, 2008. The person(s) controlling the copyright in some of this + material may not have granted the IETF Trust the right to allow + modifications of such material outside the IETF Standards Process. + Without obtaining an adequate license from the person(s) controlling + the copyright in such materials, this document may not be modified + outside the IETF Standards Process, and derivative works of it may + not be created outside the IETF Standards Process, except to format + it for publication as an RFC or to translate it into languages other + than English. + +Table of Contents + + 1. Introduction ....................................................3 + 2. Notational Conventions ..........................................3 + 3. Links ...........................................................4 + 4. Link Relation Types .............................................5 + 4.1. Registered Relation Types ..................................5 + 4.2. Extension Relation Types ...................................6 + 5. The Link Header Field ...........................................6 + 5.1. Target IRI .................................................7 + 5.2. Context IRI ................................................7 + 5.3. Relation Type ..............................................8 + 5.4. Target Attributes ..........................................8 + 5.5. Examples ...................................................9 + 6. IANA Considerations ............................................10 + 6.1. Link HTTP Header Registration .............................10 + 6.2. Link Relation Type Registry ...............................10 + 6.2.1. Registering New Link Relation Types ................11 + 6.2.2. Initial Registry Contents ..........................12 + 6.3. Link Relation Application Data Registry ...................16 + 7. Security Considerations ........................................17 + 8. Internationalisation Considerations ............................18 + 9. References .....................................................18 + 9.1. Normative References ......................................18 + 9.2. Informative References ....................................19 + Appendix A. Notes on Using the Link Header with the HTML4 + Format ...............................................21 + Appendix B. Notes on Using the Link Header with the Atom + Format ...............................................22 + Appendix C. Acknowledgements .....................................23 + + + + + + + + + +Nottingham Standards Track [Page 2] + +RFC 5988 Web Linking October 2010 + + +1. Introduction + + A means of indicating the relationships between resources on the Web, + as well as indicating the type of those relationships, has been + available for some time in HTML [W3C.REC-html401-19991224], and more + recently in Atom [RFC4287]. These mechanisms, although conceptually + similar, are separately specified. However, links between resources + need not be format specific; it can be useful to have typed links + that are independent of their serialisation, especially when a + resource has representations in multiple formats. + + To this end, this document defines a framework for typed links that + isn't specific to a particular serialisation or application. It does + so by redefining the link relation registry established by Atom to + have a broader domain, and adding to it the relations that are + defined by HTML. + + Furthermore, an HTTP header field for conveying typed links was + defined in Section 19.6.2.4 of [RFC2068], but removed from [RFC2616], + due to a lack of implementation experience. Since then, it has been + implemented in some User Agents (e.g., for stylesheets), and several + additional use cases have surfaced. + + Because it was removed, the status of the Link header is unclear, + leading some to consider minting new application-specific HTTP + headers instead of reusing it. This document addresses this by re- + specifying the Link header as one such serialisation, with updated + but backwards-compatible syntax. + +2. Notational Conventions + + The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", + "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this + document are to be interpreted as described in BCP 14, [RFC2119], as + scoped to those conformance targets. + + This document uses the Augmented Backus-Naur Form (ABNF) notation of + [RFC2616], and explicitly includes the following rules from it: + quoted-string, token, SP (space), LOALPHA, DIGIT. + + Additionally, the following rules are included from [RFC3986]: URI + and URI-Reference; from [RFC4288]: type-name and subtype-name; from + [W3C.REC-html401-19991224]: MediaDesc; from [RFC5646]: Language-Tag; + and from [RFC5987], ext-value and parmname. + + + + + + + +Nottingham Standards Track [Page 3] + +RFC 5988 Web Linking October 2010 + + +3. Links + + In this specification, a link is a typed connection between two + resources that are identified by Internationalised Resource + Identifiers (IRIs) [RFC3987], and is comprised of: + + o A context IRI, + + o a link relation type (Section 4), + + o a target IRI, and + + o optionally, target attributes. + + A link can be viewed as a statement of the form "{context IRI} has a + {relation type} resource at {target IRI}, which has {target + attributes}". + + Note that in the common case, the context IRI will also be a URI + [RFC3986], because many protocols (such as HTTP) do not support + dereferencing IRIs. Likewise, the target IRI will be converted to a + URI (see [RFC3987], Section 3.1) in serialisations that do not + support IRIs (e.g., the Link header). + + This specification does not place restrictions on the cardinality of + links; there can be multiple links to and from a particular IRI, and + multiple links of different types between two given IRIs. Likewise, + the relative ordering of links in any particular serialisation, or + between serialisations (e.g., the Link header and in-content links) + is not specified or significant in this specification; applications + that wish to consider ordering significant can do so. + + Target attributes are a set of key/value pairs that describe the link + or its target; for example, a media type hint. This specification + does not attempt to coordinate their names or use, but does provide + common target attributes for use in the Link HTTP header. + + Finally, this specification does not define a general syntax for + expressing links, nor does it mandate a specific context for any + given link; it is expected that serialisations of links will specify + both aspects. One such serialisation is communication of links + through HTTP headers, specified in Section 5. + + + + + + + + + +Nottingham Standards Track [Page 4] + +RFC 5988 Web Linking October 2010 + + +4. Link Relation Types + + In the simplest case, a link relation type identifies the semantics + of a link. For example, a link with the relation type "copyright" + indicates that the resource identified by the target IRI is a + statement of the copyright terms applying to the current context IRI. + + Link relation types can also be used to indicate that the target + resource has particular attributes, or exhibits particular + behaviours; for example, a "service" link implies that the identified + resource is part of a defined protocol (in this case, a service + description). + + Relation types are not to be confused with media types [RFC4288]; + they do not identify the format of the representation that results + when the link is dereferenced. Rather, they only describe how the + current context is related to another resource. + + Relation types SHOULD NOT infer any additional semantics based upon + the presence or absence of another link relation type, or its own + cardinality of occurrence. An exception to this is the combination + of the "alternate" and "stylesheet" registered relation types, which + has special meaning in HTML4 for historical reasons. + + There are two kinds of relation types: registered and extension. + +4.1. Registered Relation Types + + Well-defined relation types can be registered as tokens for + convenience and/or to promote reuse by other applications. This + specification establishes an IANA registry of such relation types; + see Section 6.2. + + Registered relation type names MUST conform to the reg-rel-type rule, + and MUST be compared character-by-character in a case-insensitive + fashion. They SHOULD be appropriate to the specificity of the + relation type; i.e., if the semantics are highly specific to a + particular application, the name should reflect that, so that more + general names are available for less specific use. + + Registered relation types MUST NOT constrain the media type of the + context IRI, and MUST NOT constrain the available representation + media types of the target IRI. However, they can specify the + behaviours and properties of the target resource (e.g., allowable + HTTP methods, request and response media types that must be + supported). + + + + + +Nottingham Standards Track [Page 5] + +RFC 5988 Web Linking October 2010 + + + Additionally, specific applications of linking may require additional + data to be included in the registry. For example, Web browsers might + want to know what kinds of links should be downloaded when they + archive a Web page; if this application-specific information is in + the registry, new link relation types can control this behaviour + without unnecessary coordination. + + To accommodate this, per-entry application data can be added to the + Link Relation Type registry, by registering it in the Link Relation + Application Data registry (Section 6.3). + +4.2. Extension Relation Types + + Applications that don't wish to register a relation type can use an + extension relation type, which is a URI [RFC3986] that uniquely + identifies the relation type. Although the URI can point to a + resource that contains a definition of the semantics of the relation + type, clients SHOULD NOT automatically access that resource to avoid + overburdening its server. + + When extension relation types are compared, they MUST be compared as + strings (after converting to URIs if serialised in a different + format, such as a Curie [W3C.CR-curie-20090116]) in a case- + insensitive fashion, character-by-character. Because of this, all- + lowercase URIs SHOULD be used for extension relations. + + Note that while extension relation types are required to be URIs, a + serialisation of links can specify that they are expressed in another + form, as long as they can be converted to URIs. + +5. The Link Header Field + + The Link entity-header field provides a means for serialising one or + more links in HTTP headers. It is semantically equivalent to the + element in HTML, as well as the atom:link feed-level element + in Atom [RFC4287]. + + + + + + + + + + + + + + + +Nottingham Standards Track [Page 6] + +RFC 5988 Web Linking October 2010 + + + Link = "Link" ":" #link-value + link-value = "<" URI-Reference ">" *( ";" link-param ) + link-param = ( ( "rel" "=" relation-types ) + | ( "anchor" "=" <"> URI-Reference <"> ) + | ( "rev" "=" relation-types ) + | ( "hreflang" "=" Language-Tag ) + | ( "media" "=" ( MediaDesc | ( <"> MediaDesc <"> ) ) ) + | ( "title" "=" quoted-string ) + | ( "title*" "=" ext-value ) + | ( "type" "=" ( media-type | quoted-mt ) ) + | ( link-extension ) ) + link-extension = ( parmname [ "=" ( ptoken | quoted-string ) ] ) + | ( ext-name-star "=" ext-value ) + ext-name-star = parmname "*" ; reserved for RFC2231-profiled + ; extensions. Whitespace NOT + ; allowed in between. + ptoken = 1*ptokenchar + ptokenchar = "!" | "#" | "$" | "%" | "&" | "'" | "(" + | ")" | "*" | "+" | "-" | "." | "/" | DIGIT + | ":" | "<" | "=" | ">" | "?" | "@" | ALPHA + | "[" | "]" | "^" | "_" | "`" | "{" | "|" + | "}" | "~" + media-type = type-name "/" subtype-name + quoted-mt = <"> media-type <"> + relation-types = relation-type + | <"> relation-type *( 1*SP relation-type ) <"> + relation-type = reg-rel-type | ext-rel-type + reg-rel-type = LOALPHA *( LOALPHA | DIGIT | "." | "-" ) + ext-rel-type = URI + +5.1. Target IRI + + Each link-value conveys one target IRI as a URI-Reference (after + conversion to one, if necessary; see [RFC3987], Section 3.1) inside + angle brackets ("<>"). If the URI-Reference is relative, parsers + MUST resolve it as per [RFC3986], Section 5. Note that any base IRI + from the message's content is not applied. + +5.2. Context IRI + + By default, the context of a link conveyed in the Link header field + is the IRI of the requested resource. + + When present, the anchor parameter overrides this with another URI, + such as a fragment of this resource, or a third resource (i.e., when + the anchor value is an absolute URI). If the anchor parameter's + + + + + +Nottingham Standards Track [Page 7] + +RFC 5988 Web Linking October 2010 + + + value is a relative URI, parsers MUST resolve it as per [RFC3986], + Section 5. Note that any base URI from the body's content is not + applied. + + Consuming implementations can choose to ignore links with an anchor + parameter. For example, the application in use may not allow the + context IRI to be assigned to a different resource. In such cases, + the entire link is to be ignored; consuming implementations MUST NOT + process the link without applying the anchor. + + Note that depending on HTTP status code and response headers, the + context IRI might be "anonymous" (i.e., no context IRI is available). + For instance, this is the case on a 404 response to a GET request. + +5.3. Relation Type + + The relation type of a link is conveyed in the "rel" parameter's + value. The "rel" parameter MUST NOT appear more than once in a given + link-value; occurrences after the first MUST be ignored by parsers. + + The "rev" parameter has been used in the past to indicate that the + semantics of the relationship are in the reverse direction. That is, + a link from A to B with REL="X" expresses the same relationship as a + link from B to A with REV="X". "rev" is deprecated by this + specification because it often confuses authors and readers; in most + cases, using a separate relation type is preferable. + + Note that extension relation types are REQUIRED to be absolute URIs + in Link headers, and MUST be quoted if they contain a semicolon (";") + or comma (",") (as these characters are used as delimiters in the + header itself). + +5.4. Target Attributes + + The "hreflang", "media", "title", "title*", "type", and any link- + extension link-params are considered to be target attributes for the + link. + + The "hreflang" parameter, when present, is a hint indicating what the + language of the result of dereferencing the link should be. Note + that this is only a hint; for example, it does not override the + Content-Language header of a HTTP response obtained by actually + following the link. Multiple "hreflang" parameters on a single link- + value indicate that multiple languages are available from the + indicated resource. + + + + + + +Nottingham Standards Track [Page 8] + +RFC 5988 Web Linking October 2010 + + + The "media" parameter, when present, is used to indicate intended + destination medium or media for style information (see + [W3C.REC-html401-19991224], Section 6.13). Note that this may be + updated by [W3C.CR-css3-mediaqueries-20090915]). Its value MUST be + quoted if it contains a semicolon (";") or comma (","), and there + MUST NOT be more than one "media" parameter in a link-value. + + The "title" parameter, when present, is used to label the destination + of a link such that it can be used as a human-readable identifier + (e.g., a menu entry) in the language indicated by the Content- + Language header (if present). The "title" parameter MUST NOT appear + more than once in a given link-value; occurrences after the first + MUST be ignored by parsers. + + The "title*" parameter can be used to encode this label in a + different character set, and/or contain language information as per + [RFC5987]. The "title*" parameter MUST NOT appear more than once in + a given link-value; occurrences after the first MUST be ignored by + parsers. If the parameter does not contain language information, its + language is indicated by the Content-Language header (when present). + + If both the "title" and "title*" parameters appear in a link-value, + processors SHOULD use the "title*" parameter's value. + + The "type" parameter, when present, is a hint indicating what the + media type of the result of dereferencing the link should be. Note + that this is only a hint; for example, it does not override the + Content-Type header of a HTTP response obtained by actually following + the link. There MUST NOT be more than one type parameter in a link- + value. + +5.5. Examples + + For example: + + Link: ; rel="previous"; + title="previous chapter" + + indicates that "chapter2" is previous to this resource in a logical + navigation path. + + Similarly, + + Link: ; rel="http://example.net/foo" + + indicates that the root resource ("/") is related to this resource + with the extension relation type "http://example.net/foo". + + + + +Nottingham Standards Track [Page 9] + +RFC 5988 Web Linking October 2010 + + + The example below shows an instance of the Link header encoding + multiple links, and also the use of RFC 2231 encoding to encode both + non-ASCII characters and language information. + + Link: ; + rel="previous"; title*=UTF-8'de'letztes%20Kapitel, + ; + rel="next"; title*=UTF-8'de'n%c3%a4chstes%20Kapitel + + Here, both links have titles encoded in UTF-8, use the German + language ("de"), and the second link contains the Unicode code point + U+00E4 ("LATIN SMALL LETTER A WITH DIAERESIS"). + + Note that link-values can convey multiple links between the same + target and context IRIs; for example: + + Link: ; + rel="start http://example.net/relation/other" + + Here, the link to "http://example.org/" has the registered relation + type "start" and the extension relation type + "http://example.net/relation/other". + +6. IANA Considerations + +6.1. Link HTTP Header Registration + + This specification updates the Message Header registry entry for + "Link" in HTTP [RFC3864] to refer to this document. + + Header field: Link + Applicable protocol: http + Status: standard + Author/change controller: + IETF (iesg@ietf.org) + Internet Engineering Task Force + Specification document(s): + [RFC5988] + +6.2. Link Relation Type Registry + + This specification establishes the Link Relation Type registry, and + updates Atom [RFC4287] to refer to it in place of the "Registry of + Link Relations". + + The underlying registry data (e.g., the XML file) must include + Simplified BSD License text as described in Section 4.e of the Trust + Legal Provisions (). + + + +Nottingham Standards Track [Page 10] + +RFC 5988 Web Linking October 2010 + + +6.2.1. Registering New Link Relation Types + + Relation types are registered on the advice of a Designated Expert + (appointed by the IESG or their delegate), with a Specification + Required (using terminology from [RFC5226]). + + The requirements for registered relation types are described in + Section 4.1. + + Registration requests consist of the completed registration template + below, typically published in an RFC or Open Standard (in the sense + described by [RFC2026], Section 7). However, to allow for the + allocation of values prior to publication, the Designated Expert may + approve registration once they are satisfied that a specification + will be published. + + Note that relation types can be registered by third parties, if the + Designated Expert determines that an unregistered relation type is + widely deployed and not likely to be registered in a timely manner. + + The registration template is: + + o Relation Name: + + o Description: + + o Reference: + + o Notes: [optional] + + o Application Data: [optional] + + Registration requests should be sent to the link-relations@ietf.org + mailing list, marked clearly in the subject line (e.g., "NEW RELATION + - example" to register an "example" relation type). + + Within at most 14 days of the request, the Designated Expert(s) will + either approve or deny the registration request, communicating this + decision to the review list and IANA. Denials should include an + explanation and, if applicable, suggestions as to how to make the + request successful. + + Decisions (or lack thereof) made by the Designated Expert can be + first appealed to Application Area Directors (contactable using + app-ads@tools.ietf.org email address or directly by looking up their + email addresses on http://www.iesg.org/ website) and, if the + appellant is not satisfied with the response, to the full IESG (using + the iesg@iesg.org mailing list). + + + +Nottingham Standards Track [Page 11] + +RFC 5988 Web Linking October 2010 + + + IANA should only accept registry updates from the Designated + Expert(s), and should direct all requests for registration to the + review mailing list. + +6.2.2. Initial Registry Contents + + The Link Relation Type registry's initial contents are: + + o Relation Name: alternate + o Description: Designates a substitute for the link's context. + o Reference: [W3C.REC-html401-19991224] + + o Relation Name: appendix + o Description: Refers to an appendix. + o Reference: [W3C.REC-html401-19991224] + + o Relation Name: bookmark + o Description: Refers to a bookmark or entry point. + o Reference: [W3C.REC-html401-19991224] + + o Relation Name: chapter + o Description: Refers to a chapter in a collection of resources. + o Reference: [W3C.REC-html401-19991224] + + o Relation Name: contents + o Description: Refers to a table of contents. + o Reference: [W3C.REC-html401-19991224] + + o Relation Name: copyright + o Description: Refers to a copyright statement that applies to the + link's context. + o Reference: [W3C.REC-html401-19991224] + + o Relation Name: current + o Description: Refers to a resource containing the most recent + item(s) in a collection of resources. + o Reference: [RFC5005] + + o Relation Name: describedby + o Description: Refers to a resource providing information about the + link's context. + o Documentation: + + o Relation Name: edit + o Description: Refers to a resource that can be used to edit the + link's context. + o Reference: [RFC5023] + + + + +Nottingham Standards Track [Page 12] + +RFC 5988 Web Linking October 2010 + + + o Relation Name: edit-media + o Description: Refers to a resource that can be used to edit media + associated with the link's context. + o Reference: [RFC5023] + + o Relation Name: enclosure + o Description: Identifies a related resource that is potentially + large and might require special handling. + o Reference: [RFC4287] + + o Relation Name: first + o Description: An IRI that refers to the furthest preceding resource + in a series of resources. + o Reference: [RFC5988] + o Notes: this relation type registration did not indicate a + reference. Originally requested by Mark Nottingham in December + 2004. + + o Relation Name: glossary + o Description: Refers to a glossary of terms. + o Reference: [W3C.REC-html401-19991224] + + o Relation Name: help + o Description: Refers to a resource offering help (more information, + links to other sources information, etc.) + o Reference: [W3C.REC-html401-19991224] + + o Relation Name: hub + o Description: Refers to a hub that enables registration for + notification of updates to the context. + o Reference: + o Notes: this relation type was requested by Brett Slatkin. + + o Relation Name: index + o Description: Refers to an index. + o Reference: [W3C.REC-html401-19991224] + + o Relation Name: last + o Description: An IRI that refers to the furthest following resource + in a series of resources. + o Reference: [RFC5988] + o Notes: this relation type registration did not indicate a + reference. Originally requested by Mark Nottingham in December + 2004. + + + + + + +Nottingham Standards Track [Page 13] + +RFC 5988 Web Linking October 2010 + + + o Relation Name: latest-version + o Description: Points to a resource containing the latest (e.g., + current) version of the context. + o Reference: [RFC5829] + + o Relation Name: license + o Description: Refers to a license associated with the link's + context. + o Reference: [RFC4946] + + o Relation Name: next + o Description: Refers to the next resource in a ordered series of + resources. + o Reference: [W3C.REC-html401-19991224] + + o Relation Name: next-archive + o Description: Refers to the immediately following archive resource. + o Reference: [RFC5005] + + o Relation Name: payment + o Description: indicates a resource where payment is accepted. + o Reference: [RFC5988] + o Notes: this relation type registration did not indicate a + reference. Requested by Joshua Kinberg and Robert Sayre. It is + meant as a general way to facilitate acts of payment, and thus + this specification makes no assumptions on the type of payment or + transaction protocol. Examples may include a Web page where + donations are accepted or where goods and services are available + for purchase. rel="payment" is not intended to initiate an + automated transaction. In Atom documents, a link element with a + rel="payment" attribute may exist at the feed/channel level and/or + the entry/item level. For example, a rel="payment" link at the + feed/channel level may point to a "tip jar" URI, whereas an entry/ + item containing a book review may include a rel="payment" link + that points to the location where the book may be purchased + through an online retailer. + + o Relation Name: prev + o Description: Refers to the previous resource in an ordered series + of resources. Synonym for "previous". + o Reference: [W3C.REC-html401-19991224] + + o Relation Name: predecessor-version + o Description: Points to a resource containing the predecessor + version in the version history. + o Reference: [RFC5829] + + + + + +Nottingham Standards Track [Page 14] + +RFC 5988 Web Linking October 2010 + + + o Relation Name: previous + o Description: Refers to the previous resource in an ordered series + of resources. Synonym for "prev". + o Reference: [W3C.REC-html401-19991224] + + o Relation Name: prev-archive + o Description: Refers to the immediately preceding archive resource. + o Reference: [RFC5005] + + o Relation Name: related + o Description: Identifies a related resource. + o Reference: [RFC4287] + + o Relation Name: replies + o Description: Identifies a resource that is a reply to the context + of the link. + o Reference: [RFC4685] + + o Relation Name: section + o Description: Refers to a section in a collection of resources. + o Reference: [W3C.REC-html401-19991224] + + o Relation Name: self + o Description: Conveys an identifier for the link's context. + o Reference: [RFC4287] + + o Relation Name: service + o Description: Indicates a URI that can be used to retrieve a + service document. + o Reference: [RFC5023] + o Notes: When used in an Atom document, this relation type specifies + Atom Publishing Protocol service documents by default. Requested + by James Snell. + + o Relation Name: start + o Description: Refers to the first resource in a collection of + resources. + o Reference: [W3C.REC-html401-19991224] + + o Relation Name: stylesheet + o Description: Refers to an external style sheet. + o Reference: [W3C.REC-html401-19991224] + + o Relation Name: subsection + o Description: Refers to a resource serving as a subsection in a + collection of resources. + o Reference: [W3C.REC-html401-19991224] + + + + +Nottingham Standards Track [Page 15] + +RFC 5988 Web Linking October 2010 + + + o Relation Name: successor-version + o Description: Points to a resource containing the successor version + in the version history. + o Reference: [RFC5829] + + o Relation Name: up + o Description: Refers to a parent document in a hierarchy of + documents. + o Reference: [RFC5988] + o Notes: this relation type registration did not indicate a + reference. Requested by Noah Slater. + + o Relation Name: version-history + o Description: points to a resource containing the version history + for the context. + o Reference: [RFC5829] + + o Relation Name: via + o Description: Identifies a resource that is the source of the + information in the link's context. + o Reference: [RFC4287] + + o Relation Name: working-copy + o Description: Points to a working copy for this resource. + o Reference: [RFC5829] + + o Relation Name: working-copy-of + o Description: Points to the versioned resource from which this + working copy was obtained. + o Reference: [RFC5829] + +6.3. Link Relation Application Data Registry + + This specification also establishes the Link Relation Application + Field registry, to allow entries in the Link Relation Type registry + to be extended with application-specific data (hereafter, "app data") + specific to all instances of a given link relation type. + + Application data is registered on the advice of a Designated Expert + (appointed by the IESG or their delegate), with a Specification + Required (using terminology from [RFC5226]). + + + + + + + + + + +Nottingham Standards Track [Page 16] + +RFC 5988 Web Linking October 2010 + + + Registration requests consist of the completed registration template + below: + + o Application Name: + + o Description: + + o Default Value: + + o Notes: [optional] + + The Description SHOULD identify the value space of the app data. The + Default Value MUST be appropriate to entries to which the app data + does not apply. + + Entries that pre-date the addition of app data will automatically be + considered to have the default value for that app data; if there are + exceptions, the modification of such entries should be coordinated by + the Designated Expert(s), in consultation with the author of the + proposed app data as well as the registrant of the existing entry (if + possible). + + Registration requests should be sent to the link-relations@ietf.org + mailing list, marked clearly in the subject line (e.g., "NEW APP DATA + - example" to register "example" app data). + + Within at most 14 days of the request, the Designated Expert will + either approve or deny the registration request, communicating this + decision to the review list. Denials should include an explanation + and, if applicable, suggestions as to how to make the request + successful. Registration requests that are undetermined for a period + longer than 21 days can be brought to the IESG's attention (using the + iesg@iesg.org mailing list) for resolution. + + When a registration request is successful, the Designated Expert will + forward it to IANA for publication. IANA should only accept registry + updates from the Designated Expert(s), and should direct all requests + for registration to the review mailing list. + +7. Security Considerations + + The content of the Link header field is not secure, private or + integrity-guaranteed, and due caution should be exercised when using + it. Use of Transport Layer Security (TLS) with HTTP ([RFC2818] and + [RFC2817]) is currently the only end-to-end way to provide such + protection. + + + + + +Nottingham Standards Track [Page 17] + +RFC 5988 Web Linking October 2010 + + + Applications that take advantage of typed links should consider the + attack vectors opened by automatically following, trusting, or + otherwise using links gathered from HTTP headers. In particular, + Link headers that use the "anchor" parameter to associate a link's + context with another resource should be treated with due caution. + + The Link entity-header field makes extensive use of IRIs and URIs. + See [RFC3987] for security considerations relating to IRIs. See + [RFC3986] for security considerations relating to URIs. See + [RFC2616] for security considerations relating to HTTP headers. + +8. Internationalisation Considerations + + Target IRIs may need to be converted to URIs in order to express them + in serialisations that do not support IRIs. This includes the Link + HTTP header. + + Similarly, the anchor parameter of the Link header does not support + IRIs, and therefore IRIs must be converted to URIs before inclusion + there. + + Relation types are defined as URIs, not IRIs, to aid in their + comparison. It is not expected that they will be displayed to end + users. + +9. References + +9.1. Normative References + + [RFC2026] Bradner, S., "The Internet Standards Process -- Revision + 3", BCP 9, RFC 2026, October 1996. + + [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate + Requirement Levels", BCP 14, RFC 2119, March 1997. + + [RFC2616] Fielding, R., Gettys, J., Mogul, J., Frystyk, H., + Masinter, L., Leach, P., and T. Berners-Lee, "Hypertext + Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999. + + [RFC3864] Klyne, G., Nottingham, M., and J. Mogul, "Registration + Procedures for Message Header Fields", BCP 90, RFC 3864, + September 2004. + + [RFC3986] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform + Resource Identifier (URI): Generic Syntax", STD 66, + RFC 3986, January 2005. + + + + + +Nottingham Standards Track [Page 18] + +RFC 5988 Web Linking October 2010 + + + [RFC3987] Duerst, M. and M. Suignard, "Internationalized Resource + Identifiers (IRIs)", RFC 3987, January 2005. + + [RFC4288] Freed, N. and J. Klensin, "Media Type Specifications and + Registration Procedures", BCP 13, RFC 4288, December 2005. + + [RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an + IANA Considerations Section in RFCs", BCP 26, RFC 5226, + May 2008. + + [RFC5646] Phillips, A. and M. Davis, "Tags for Identifying + Languages", BCP 47, RFC 5646, September 2009. + + [RFC5987] Reschke, J., "Character Set and Language Encoding for + Hypertext Transfer Protocol (HTTP) Header Field + Parameters", RFC 5987, August 2010. + +9.2. Informative References + + [RFC2068] Fielding, R., Gettys, J., Mogul, J., Nielsen, H., and T. + Berners-Lee, "Hypertext Transfer Protocol -- HTTP/1.1", + RFC 2068, January 1997. + + [RFC2817] Khare, R. and S. Lawrence, "Upgrading to TLS Within + HTTP/1.1", RFC 2817, May 2000. + + [RFC2818] Rescorla, E., "HTTP Over TLS", RFC 2818, May 2000. + + [RFC4287] Nottingham, M., Ed. and R. Sayre, Ed., "The Atom + Syndication Format", RFC 4287, December 2005. + + [RFC4685] Snell, J., "Atom Threading Extensions", RFC 4685, + September 2006. + + [RFC4946] Snell, J., "Atom License Extension", RFC 4946, July 2007. + + [RFC5005] Nottingham, M., "Feed Paging and Archiving", RFC 5005, + September 2007. + + [RFC5023] Gregorio, J. and B. de hOra, "The Atom Publishing + Protocol", RFC 5023, October 2007. + + [RFC5829] Brown, A., Clemm, G., and J. Reschke, "Link Relation Types + for Simple Version Navigation between Web Resources", + RFC 5829, April 2010. + + + + + + +Nottingham Standards Track [Page 19] + +RFC 5988 Web Linking October 2010 + + + [W3C.CR-css3-mediaqueries-20090915] + van Kesteren, A., Glazman, D., Lie, H., and T. Celik, + "Media Queries", W3C Candidate Recommendation CR-css3- + mediaqueries-20090915, September 2009, + . + + Latest version available at + . + + [W3C.CR-curie-20090116] + Birbeck, M. and S. McCarron, "CURIE Syntax 1.0", W3C + Candidate Recommendation CR-curie-20090116, January 2009, + . + + Latest version available at . + + [W3C.REC-html401-19991224] + Le Hors, A., Raggett, D., and I. Jacobs, "HTML 4.01 + Specification", W3C Recommendation REC-html401-19991224, + December 1999, + . + + Latest version available at + . + + [W3C.REC-rdfa-syntax-20081014] + Adida, B., Birbeck, M., McCarron, S., and S. Pemberton, + "RDFa in XHTML: Syntax and Processing", W3C + Recommendation REC-rdfa-syntax-20081014, October 2008, + . + + Latest version available at + . + + [W3C.REC-xhtml-basic-20080729] + Baker, M., Ishikawa, M., Stark, P., Matsui, S., Wugofski, + T., and T. Yamakami, "XHTML[TM] Basic 1.1", W3C + Recommendation REC-xhtml-basic-20080729, July 2008, + . + + Latest version available at + . + + + + + + + + +Nottingham Standards Track [Page 20] + +RFC 5988 Web Linking October 2010 + + +Appendix A. Notes on Using the Link Header with the HTML4 Format + + HTML motivated the original syntax of the Link header, and many of + the design decisions in this document are driven by a desire to stay + compatible with these uses. + + In HTML4, the link element can be mapped to links as specified here + by using the "href" attribute for the target URI, and "rel" to convey + the relation type, as in the Link header. The context of the link is + the URI associated with the entire HTML document. + + All of the link relation types defined by HTML4 have been included in + the Link Relation Type registry, so they can be used without + modification. However, there are several potential ways to serialise + extension relation types into HTML4, including + + o As absolute URIs, + + o using the document-wide "profile" attribute's URI as a prefix for + relation types, or + + o using the RDFa [W3C.REC-rdfa-syntax-20081014] convention of + mapping token prefixes to URIs (in a manner similar to XML name + spaces) (note that RDFa is only defined to work in XHTML + [W3C.REC-xhtml-basic-20080729], but is sometimes used in HTML4). + + Individual applications of linking will therefore need to define how + their extension links should be serialised into HTML4. + + Surveys of existing HTML content have shown that unregistered link + relation types that are not URIs are (perhaps inevitably) common. + Consuming HTML implementations should not consider such unregistered + short links to be errors, but rather relation types with a local + scope (i.e., their meaning is specific and perhaps private to that + document). + + HTML4 also defines several attributes on links that are not + explicitly defined by the Link header. These attributes can be + serialised as link-extensions to maintain fidelity. + + Finally, the HTML4 specification gives a special meaning when the + "alternate" and "stylesheet" relation types coincide in the same + link. Such links should be serialised in the Link header using a + single list of relation-types (e.g., rel="alternate stylesheet") to + preserve this relationship. + + + + + + +Nottingham Standards Track [Page 21] + +RFC 5988 Web Linking October 2010 + + +Appendix B. Notes on Using the Link Header with the Atom Format + + Atom conveys links in the atom:link element, with the "href" + attribute indicating the target IRI and the "rel" attribute + containing the relation type. The context of the link is either a + feed IRI or an entry ID, depending on where it appears; generally, + feed-level links are obvious candidates for transmission as a Link + header. + + When serialising an atom:link into a Link header, it is necessary to + convert target IRIs (if used) to URIs. + + Atom defines extension relation types in terms of IRIs. This + specification re-defines them as URIs, to simplify and reduce errors + in their comparison. + + Atom allows registered link relation types to be serialised as + absolute URIs. Such relation types SHOULD be converted to the + appropriate registered form (e.g., + "http://www.iana.org/assignments/relation/self" to "self") so that + they are not mistaken for extension relation types. + + Furthermore, Atom link relation types are always compared in a case- + sensitive fashion; therefore, registered link relation types SHOULD + be converted to their registered form (usually, lowercase) when + serialised in an Atom document. + + Note also that while the Link header allows multiple relations to be + serialised in a single link, atom:link does not. In this case, a + single link-value may map to several atom:link elements. + + As with HTML, atom:link defines some attributes that are not + explicitly mirrored in the Link header syntax, but they can also be + used as link-extensions to maintain fidelity. + + + + + + + + + + + + + + + + + +Nottingham Standards Track [Page 22] + +RFC 5988 Web Linking October 2010 + + +Appendix C. Acknowledgements + + This specification lifts the idea and definition for the Link header + from RFC 2068; credit for it belongs entirely to the authors of and + contributors to that document. The link relation type registrations + themselves are sourced from several documents; see the applicable + references. + + The author would like to thank the many people who commented upon, + encouraged and gave feedback to this specification, especially + including Frank Ellermann, Roy Fielding, Eran Hammer-Lahav, and + Julian Reschke. + +Author's Address + + Mark Nottingham + + EMail: mnot@mnot.net + URI: http://www.mnot.net/ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +Nottingham Standards Track [Page 23] + diff --git a/docs/rfc8288.txt b/docs/rfc8288.txt new file mode 100644 index 0000000..b2a517e --- /dev/null +++ b/docs/rfc8288.txt @@ -0,0 +1,1347 @@ + + + + + + +Internet Engineering Task Force (IETF) M. Nottingham +Request for Comments: 8288 October 2017 +Obsoletes: 5988 +Category: Standards Track +ISSN: 2070-1721 + + + Web Linking + +Abstract + + This specification defines a model for the relationships between + resources on the Web ("links") and the type of those relationships + ("link relation types"). + + It also defines the serialisation of such links in HTTP headers with + the Link header field. + +Status of This Memo + + This is an Internet Standards Track document. + + This document is a product of the Internet Engineering Task Force + (IETF). It represents the consensus of the IETF community. It has + received public review and has been approved for publication by the + Internet Engineering Steering Group (IESG). Further information on + Internet Standards is available in Section 2 of RFC 7841. + + Information about the current status of this document, any errata, + and how to provide feedback on it may be obtained at + https://www.rfc-editor.org/info/rfc8288. + + + + + + + + + + + + + + + + + + + + +Nottingham Standards Track [Page 1] + +RFC 8288 Web Linking October 2017 + + +Copyright Notice + + Copyright (c) 2017 IETF Trust and the persons identified as the + document authors. All rights reserved. + + This document is subject to BCP 78 and the IETF Trust's Legal + Provisions Relating to IETF Documents + (https://trustee.ietf.org/license-info) in effect on the date of + publication of this document. Please review these documents + carefully, as they describe your rights and restrictions with respect + to this document. Code Components extracted from this document must + include Simplified BSD License text as described in Section 4.e of + the Trust Legal Provisions and are provided without warranty as + described in the Simplified BSD License. + + This document may contain material from IETF Documents or IETF + Contributions published or made publicly available before November + 10, 2008. The person(s) controlling the copyright in some of this + material may not have granted the IETF Trust the right to allow + modifications of such material outside the IETF Standards Process. + Without obtaining an adequate license from the person(s) controlling + the copyright in such materials, this document may not be modified + outside the IETF Standards Process, and derivative works of it may + not be created outside the IETF Standards Process, except to format + it for publication as an RFC or to translate it into languages other + than English. + + + + + + + + + + + + + + + + + + + + + + + + + +Nottingham Standards Track [Page 2] + +RFC 8288 Web Linking October 2017 + + +Table of Contents + + 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 4 + 1.1. Notational Conventions . . . . . . . . . . . . . . . . . 4 + 1.2. Conformance and Error Handling . . . . . . . . . . . . . 4 + 2. Links . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 + 2.1. Link Relation Types . . . . . . . . . . . . . . . . . . . 6 + 2.1.1. Registered Relation Types . . . . . . . . . . . . . . 6 + 2.1.2. Extension Relation Types . . . . . . . . . . . . . . 8 + 2.2. Target Attributes . . . . . . . . . . . . . . . . . . . . 9 + 3. Link Serialisation in HTTP Headers . . . . . . . . . . . . . 9 + 3.1. Link Target . . . . . . . . . . . . . . . . . . . . . . . 10 + 3.2. Link Context . . . . . . . . . . . . . . . . . . . . . . 10 + 3.3. Relation Type . . . . . . . . . . . . . . . . . . . . . . 11 + 3.4. Target Attributes . . . . . . . . . . . . . . . . . . . . 11 + 3.4.1. Serialisation-Defined Attributes . . . . . . . . . . 11 + 3.4.2. Extension Attributes . . . . . . . . . . . . . . . . 13 + 3.5. Link Header Field Examples . . . . . . . . . . . . . . . 13 + 4. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 14 + 4.1. Link HTTP Header Field Registration . . . . . . . . . . . 14 + 4.2. Link Relation Type Registry . . . . . . . . . . . . . . . 14 + 4.3. Link Relation Application Data Registry . . . . . . . . . 15 + 5. Security Considerations . . . . . . . . . . . . . . . . . . . 15 + 6. Internationalisation Considerations . . . . . . . . . . . . . 16 + 7. References . . . . . . . . . . . . . . . . . . . . . . . . . 16 + 7.1. Normative References . . . . . . . . . . . . . . . . . . 16 + 7.2. Informative References . . . . . . . . . . . . . . . . . 17 + Appendix A. Notes on Other Link Serialisations . . . . . . . . . 19 + A.1. Link Serialisation in HTML . . . . . . . . . . . . . . . 19 + A.2. Link Serialisation in Atom . . . . . . . . . . . . . . . 19 + Appendix B. Algorithms for Parsing Link Header Fields . . . . . 20 + B.1. Parsing a Header Set for Links . . . . . . . . . . . . . 20 + B.2. Parsing a Link Field Value . . . . . . . . . . . . . . . 21 + B.3. Parsing Parameters . . . . . . . . . . . . . . . . . . . 22 + B.4. Parsing a Quoted String . . . . . . . . . . . . . . . . . 23 + Appendix C. Changes from RFC 5988 . . . . . . . . . . . . . . . 24 + Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 24 + + + + + + + + + + + + + + +Nottingham Standards Track [Page 3] + +RFC 8288 Web Linking October 2017 + + +1. Introduction + + This specification defines a model for the relationships between + resources on the Web ("links") and the type of those relationships + ("link relation types"). + + HTML [W3C.REC-html5-20141028] and Atom [RFC4287] both have well- + defined concepts of linking; Section 2 generalises this into a + framework that encompasses linking in these formats and (potentially) + elsewhere. + + Furthermore, Section 3 defines an HTTP header field for conveying + such links. + +1.1. Notational Conventions + + The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", + "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and + "OPTIONAL" in this document are to be interpreted as described in BCP + 14 [RFC2119] [RFC8174] when, and only when, they appear in all + capitals, as shown here. + + This document uses the Augmented Backus-Naur Form (ABNF) [RFC5234] + notation of [RFC7230], including the #rule, and explicitly includes + the following rules from it: quoted-string, token, SP (space), BWS + (bad whitespace), OWS (optional whitespace), RWS (required + whitespace), LOALPHA, DIGIT. + + Additionally, the following rules are included: + + o URI and URI-Reference from [RFC3986], + o type-name and subtype-name from [RFC6838], + o media-query-list from [W3C.REC-css3-mediaqueries-20120619], and + o Language-Tag from [RFC5646]. + +1.2. Conformance and Error Handling + + The requirements regarding conformance and error handling highlighted + in [RFC7230], Section 2.5 apply to this document. + + + + + + + + + + + + +Nottingham Standards Track [Page 4] + +RFC 8288 Web Linking October 2017 + + +2. Links + + In this specification, a link is a typed connection between two + resources and is comprised of: + + o a link context, + o a link relation type (Section 2.1), + o a link target, and + o optionally, target attributes (Section 2.2). + + A link can be viewed as a statement of the form "link context has a + link relation type resource at link target, which has target + attributes". + + For example, "https://www.example.com/" has a "canonical" resource at + "https://example.com", which has a "type" of "text/html". + + Link contexts and link targets are both Internationalized Resource + Identifiers (IRIs) [RFC3987]. However, in the common case, the link + context will also be a URI [RFC3986], because many protocols (such as + HTTP) do not support dereferencing IRIs. Likewise, the link target + will sometimes be converted to a URI (see [RFC3987], Section 3.1) in + serialisations that do not support IRIs (such as the Link header + field defined in Section 3). + + This specification does not place restrictions on the cardinality of + links; there can be multiple links to and from a particular target + and multiple links of the same or different types between a given + context and target. Likewise, the relative ordering of links in any + particular serialisation, or between serialisations (e.g., the Link + header field and in-content links), is not specified or significant + in this specification; applications that wish to consider ordering + significant can do so. + + Links are conveyed in link serialisations; they are the "bytes on the + wire", and can occur in various forms. For example, Atom [RFC4287] + and HTML [W3C.REC-html5-20141028] both defined serialisations of + links into their respective formats, and Section 3 defines how to + serialise links in HTTP header fields. + + This specification does not define a general syntax for links across + different serialisations, nor does it mandate a specific context for + any given link; it is expected that serialisations of links will + specify both aspects. + + Finally, links are used by link applications. Generally, an + application will define the link relation type(s) it uses, along with + the serialisation(s) that they might occur within. For example, the + + + +Nottingham Standards Track [Page 5] + +RFC 8288 Web Linking October 2017 + + + application "Web browsing" looks for the "stylesheet" link relation + type in the HTML link serialisation (and optionally in the Link + header field), whereas the application "AtomPub" uses the "edit" and + "edit-media" link relations in the Atom serialisation. + +2.1. Link Relation Types + + In the simplest case, a link relation type identifies the semantics + of a link. For example, a link with the relation type "copyright" + indicates that the current link context has a copyright resource at + the link target. + + Link relation types can also be used to indicate that the target + resource has particular attributes, or exhibits particular + behaviours; for example, a "service" link implies that the link + target can be used as part of a defined protocol (in this case, a + service description). + + Relation types are not to be confused with media types [RFC2046]; + they do not identify the format of the representation that results + when the link is dereferenced. Rather, they only describe how the + current context is related to another resource. + + Relation types SHOULD NOT infer any additional semantics based upon + the presence or absence of another link relation type, or its own + cardinality of occurrence. An exception to this is the combination + of the "alternate" and "stylesheet" registered relation types, which + has special meaning in HTML for historical reasons. + + There are two kinds of relation types: registered and extension. + +2.1.1. Registered Relation Types + + Well-defined relation types can be registered as tokens for + convenience and/or to promote reuse by other applications, using the + procedure in Section 2.1.1.1. + + Registered relation type names MUST conform to the reg-rel-type rule + (see Section 3.3) and MUST be compared character by character in a + case-insensitive fashion. They SHOULD be appropriate to the + specificity of the relation type; that is, if the semantics are + highly specific to a particular application, the name should reflect + that, so that more general names are available for less-specific use. + + Registered relation types MUST NOT constrain the media type of the + link context and MUST NOT constrain the available representation + media types of the link target. However, they can specify the + behaviours and properties of the target resource (e.g., allowable + + + +Nottingham Standards Track [Page 6] + +RFC 8288 Web Linking October 2017 + + + HTTP methods, and request and response media types that are required + be supported). + + Historically, registered relation types have been identified with a + URI [RFC3986] by prefixing their names with an application-defined + base URI (e.g., see Appendix A.2). This practice is NOT RECOMMENDED, + because the resulting strings will not be considered equivalent to + the registered relation types by other applications. Applications + that do use such URIs internally MUST NOT use them in link + serialisations that do not explicitly accommodate them. + +2.1.1.1. Registering Link Relation Types + + The "Link Relations" registry is located at + . Registration + requests can be made by following the instructions located there or + by sending an email to the mailing list. + + Registration requests consist of at least the following information: + + o *Relation Name*: The name of the relation type + + o *Description*: A short English description of the type's + semantics. It SHOULD be stated in terms of the relationship + between the link context and link target. + + o *Reference*: Reference to the document that specifies the link + relation type, preferably including a URI that can be used to + retrieve a copy of the document. An indication of the relevant + section(s) can also be included but is not required. + + The expert(s) can define additional fields to be collected in the + registry. + + General requirements for registered relation types are described in + Section 2.1.1. + + Registrations MUST reference a freely available, stable + specification. + + Note that relation types can be registered by third parties + (including the expert(s)), if the expert(s) determines that an + unregistered relation type is widely deployed and not likely to be + registered in a timely manner otherwise. Such registrations still + are subject to the requirements defined, including the need to + reference a specification. + + + + + +Nottingham Standards Track [Page 7] + +RFC 8288 Web Linking October 2017 + + +2.1.1.2. Registration Request Processing + + Relation types are registered using the Specification Required policy + (see Section 4.6 of [RFC8126]), which implies review and approval by + a designated expert. + + The goal of the registry is to reflect common use of links on the + Internet. Therefore, the expert(s) should be strongly biased towards + approving registrations, unless they are abusive, frivolous, not + likely to be used on the Internet, or actively harmful to the + Internet and/or the Web (not merely aesthetically displeasing or + architecturally dubious). As stated in Section 2.1.1, the expert(s) + can withhold registration of names that are too general for the + proposed application. + + The expert(s) will clearly identify any issues that cause a + registration to be refused. Advice about the semantics of a proposed + link relation type can be given, but if it does not block + registration, this should be explicitly stated. + + When a request is approved, the expert(s) will inform IANA, and the + registration will be processed. The IESG is the final arbiter of any + objection. + +2.1.2. Extension Relation Types + + Applications that don't wish to register a relation type can use an + extension relation type, which is a URI [RFC3986] that uniquely + identifies the relation type. Although the URI can point to a + resource that contains a definition of the semantics of the relation + type, clients SHOULD NOT automatically access that resource to avoid + overburdening its server. + + The URI used for an extension relation type SHOULD be under the + control of the person or party defining it or be delegated to them. + + When extension relation types are compared, they MUST be compared as + strings (after converting to URIs if serialised in a different + format) in a case-insensitive fashion, character by character. + Because of this, all-lowercase URIs SHOULD be used for extension + relations. + + Note that while extension relation types are required to be URIs, a + serialisation of links can specify that they are expressed in another + form, as long as they can be converted to URIs. + + + + + + +Nottingham Standards Track [Page 8] + +RFC 8288 Web Linking October 2017 + + +2.2. Target Attributes + + Target attributes are a list of key/value pairs that describe the + link or its target; for example, a media type hint. + + They can be defined both by individual link relation types and by + link serialisations. + + This specification does not attempt to coordinate the name of target + attributes, their cardinality, or use. Those creating and + maintaining serialisations SHOULD coordinate their target attributes + to avoid conflicts in semantics or syntax and MAY define their own + registries of target attributes. + + The names of target attributes SHOULD conform to the token rule, but + SHOULD NOT include any of the characters "%", "'", or "*", for + portability across serialisations and MUST be compared in a case- + insensitive fashion. + + Target attribute definitions SHOULD specify: + + o The serialisation of their values into Unicode or a subset + thereof, to maximise their chances of portability across link + serialisations. + o The semantics and error handling of multiple occurrences of the + target attribute on a given link. + + This specification does define target attributes for use in the Link + HTTP header field in Section 3.4. + +3. Link Serialisation in HTTP Headers + + The Link header field provides a means for serialising one or more + links into HTTP headers. + + The ABNF for the field value is: + + Link = #link-value + link-value = "<" URI-Reference ">" *( OWS ";" OWS link-param ) + link-param = token BWS [ "=" BWS ( token / quoted-string ) ] + + Note that any link-param can be generated with values using either + the token or the quoted-string syntax; therefore, recipients MUST be + able to parse both forms. In other words, the following parameters + are equivalent: + + x=y + x="y" + + + +Nottingham Standards Track [Page 9] + +RFC 8288 Web Linking October 2017 + + + Previous definitions of the Link header did not equate the token and + quoted-string forms explicitly; the title parameter was always + quoted, and the hreflang parameter was always a token. Senders + wishing to maximize interoperability will send them in those forms. + + Individual link-params specify their syntax in terms of the value + after any necessary unquoting (as per [RFC7230], Section 3.2.6). + + This specification establishes the link-params "rel", "anchor", and + "rev" (which are part of the general link model), as well as + "hreflang", "media", "title", "title*", and "type" (which are target + attributes defined by the serialisation). + +3.1. Link Target + + Each link-value conveys one target IRI as a URI-Reference (after + conversion to one, if necessary; see [RFC3987], Section 3.1) inside + angle brackets ("<>"). If the URI-Reference is relative, parsers + MUST resolve it as per [RFC3986], Section 5. Note that any base IRI + appearing in the message's content is not applied. + +3.2. Link Context + + By default, the context of a link conveyed in the Link header field + is the URL of the representation it is associated with, as defined in + [RFC7231], Section 3.1.4.1, and is serialised as a URI. + + When present, the anchor parameter overrides this with another URI, + such as a fragment of this resource, or a third resource (i.e., when + the anchor value is an absolute URI). If the anchor parameter's + value is a relative URI, parsers MUST resolve it as per [RFC3986], + Section 5. Note that any base URI from the body's content is not + applied. + + The ABNF for the "anchor" parameter's value is: + + URI-Reference ; Section 4.1 of [RFC3986] + + Link application can choose to ignore links with an anchor parameter. + For example, the application in use might not allow the link context + to be assigned to a different resource. In such cases, the entire + link is to be ignored; link applications MUST NOT process the link + without applying the anchor. + + Note that depending on HTTP status code and response headers, the + link context might be "anonymous" (i.e., no link context is + available). For example, this is the case on a 404 response to a GET + request. + + + +Nottingham Standards Track [Page 10] + +RFC 8288 Web Linking October 2017 + + +3.3. Relation Type + + The relation type of a link conveyed in the Link header field is + conveyed in the "rel" parameter's value. The rel parameter MUST be + present but MUST NOT appear more than once in a given link-value; + occurrences after the first MUST be ignored by parsers. + + The rel parameter can, however, contain multiple link relation types. + When this occurs, it establishes multiple links that share the same + context, target, and target attributes. + + The "rev" parameter has been used in the past to indicate that the + semantics of the relationship are in the reverse direction. That is, + a link from A to B with REL="X" expresses the same relationship as a + link from B to A with REV="X". rev is deprecated by this + specification because it often confuses authors and readers; in most + cases, using a separate relation type is preferable. + + The ABNF for the rel and rev parameters' values is: + + relation-type *( 1*SP relation-type ) + + where: + + relation-type = reg-rel-type / ext-rel-type + reg-rel-type = LOALPHA *( LOALPHA / DIGIT / "." / "-" ) + ext-rel-type = URI ; Section 3 of [RFC3986] + + Note that extension relation types are REQUIRED to be absolute URIs + in Link header fields and MUST be quoted when they contain characters + not allowed in tokens, such as a semicolon (";") or comma (",") (as + these characters are used as delimiters in the header field itself). + +3.4. Target Attributes + + The Link header field defines several target attributes specific to + this serialisation and also allows extension target attributes. + Target attributes are serialised in the Link header field as + parameters (see [RFC7231], Section 3.1.1.1 for the definition of + their syntax). + +3.4.1. Serialisation-Defined Attributes + + The "hreflang", "media", "title", "title*", and "type" link-params + can be translated to serialisation-defined target attributes for the + link. + + + + + +Nottingham Standards Track [Page 11] + +RFC 8288 Web Linking October 2017 + + + The "hreflang" attribute, when present, is a hint indicating what the + language of the result of dereferencing the link should be. Note + that this is only a hint; for example, it does not override the + Content-Language header field of a HTTP response obtained by actually + following the link. Multiple hreflang attributes on a single link- + value indicate that multiple languages are available from the + indicated resource. + + The ABNF for the hreflang parameter's value is: + + Language-Tag + + The "media" attribute, when present, is used to indicate intended + destination medium or media for style information (see + [W3C.REC-html5-20141028], Section 4.2.4). Its value MUST be quoted + if it contains a semicolon (";") or comma (","). There MUST NOT be + more than one media attribute in a link-value; occurrences after the + first MUST be ignored by parsers. + + The ABNF for the media parameter's value is: + + media-query-list + + The "title" attribute, when present, is used to label the destination + of a link such that it can be used as a human-readable identifier + (e.g., a menu entry) in the language indicated by the Content- + Language header field (if present). The title attribute MUST NOT + appear more than once in a given link; occurrences after the first + MUST be ignored by parsers. + + The "title*" link-param can be used to encode this attribute in a + different character set and/or contain language information as per + [RFC8187]. The title* link-param MUST NOT appear more than once in a + given link-value; occurrences after the first MUST be ignored by + parsers. If the attribute does not contain language information, its + language is indicated by the Content-Language header field (when + present). + + If both the title and title* link-params appear in a link, + applications SHOULD use the title* link-param's value for the title + attribute. + + The "type" attribute, when present, is a hint indicating what the + media type of the result of dereferencing the link should be. Note + that this is only a hint; for example, it does not override the + Content-Type header field of a HTTP response obtained by actually + + + + + +Nottingham Standards Track [Page 12] + +RFC 8288 Web Linking October 2017 + + + following the link. The type attribute MUST NOT appear more than + once in a given link-value; occurrences after the first MUST be + ignored by parsers. + + The ABNF for the type parameter's value is: + + type-name "/" subtype-name ; see Section 4.2 of [RFC6838] + +3.4.2. Extension Attributes + + Other link-params are link-extensions and are to be considered as + target attributes. + + Such target attributes MAY be defined to use the encoding in + [RFC8187] (e.g., "example" and "example*"). When both forms are + present, they SHOULD be considered to be the same target attribute; + applications SHOULD use the value of the name ending in "*" (after + [RFC8187] decoding) but MAY fall back to the other value if there is + an error in decoding it, or if they do not support decoding. + +3.5. Link Header Field Examples + + For example: + + Link: ; rel="previous"; + title="previous chapter" + + indicates that "chapter2" is previous to this resource in a logical + navigation path. + + Similarly, + + Link: ; rel="http://example.net/foo" + + indicates that the root resource ("/") is related to this resource + with the extension relation type "http://example.net/foo". + + This link: + + Link: ; rel="copyright"; anchor="#foo" + + indicates that the linked copyright terms only apply to the portion + of the document indicated by the (media type-specific) fragment + identifier "foo". + + The example below shows an instance of the Link header field encoding + multiple links and also the use of the encoding from RFC 8187 to + encode both non-ASCII characters and language information. + + + +Nottingham Standards Track [Page 13] + +RFC 8288 Web Linking October 2017 + + + Link: ; + rel="previous"; title*=UTF-8'de'letztes%20Kapitel, + ; + rel="next"; title*=UTF-8'de'n%c3%a4chstes%20Kapitel + + Here, both links have titles encoded in UTF-8, both use the German + language ("de"), and the second link contains the Unicode code point + U+00E4 ("LATIN SMALL LETTER A WITH DIAERESIS"). + + Note that link-values can convey multiple links between the same link + target and link context; for example: + + Link: ; + rel="start http://example.net/relation/other" + + Here, the link to "http://example.org/" has the registered relation + type "start" and the extension relation type + "http://example.net/relation/other". + + Finally, this header field: + + Link: ; rel="start", + ; rel="index" + + is equivalent to these: + + Link: ; rel="start" + Link: ; rel="index" + +4. IANA Considerations + +4.1. Link HTTP Header Field Registration + + This specification updates the "Message Headers" registry entry for + "Link" in HTTP [RFC3864] to refer to this document. + + Header Field Name: Link + Protocol: http + Status: standard + Reference: RFC 8288 + +4.2. Link Relation Type Registry + + This specification updates the registration procedures for the "Link + Relation Types" registry; see Section 2.1.1.1. Also, all references + to RFC 5988 in that registry have been replaced with references to + this document. + + + + +Nottingham Standards Track [Page 14] + +RFC 8288 Web Linking October 2017 + + + IANA will direct any incoming requests regarding the registry to this + document and, if defined, the processes established by the expert(s); + typically, this will mean referring them to the registry Web page. + + Note that the expert(s) is allowed (as per Section 2.1.1.1) to define + additional fields to be collected in the registry. + +4.3. Link Relation Application Data Registry + + Per this specification, IANA has removed the "Link Relation + Application Data" registry, as it has not been used, and future use + is not anticipated. + +5. Security Considerations + + The content of the Link header field is not secure, private, or + integrity-guaranteed. Use of Transport Layer Security (TLS) with + HTTP [RFC2818] is currently the only end-to-end way to provide these + properties. + + Link applications ought to consider the attack vectors opened by + automatically following, trusting, or otherwise using links gathered + from HTTP header fields. + + For example, Link header fields that use the "anchor" parameter to + associate a link's context with another resource cannot be trusted + since they are effectively assertions by a third party that could be + incorrect or malicious. Applications can mitigate this risk by + specifying that such links should be discarded unless some + relationship between the resources is established (e.g., they share + the same authority). + + Dereferencing links has a number of risks, depending on the + application in use. For example, the Referer header [RFC7231] can + expose information about the application's state (including private + information) in its value. Likewise, cookies [RFC6265] are another + mechanism that, if used, can become an attack vector. Applications + can mitigate these risks by carefully specifying how such mechanisms + should operate. + + The Link header field makes extensive use of IRIs and URIs. See + [RFC3987], Section 8 for security considerations relating to IRIs. + See [RFC3986], Section 7 for security considerations relating to + URIs. See [RFC7230], Section 9 for security considerations relating + to HTTP header fields. + + + + + + +Nottingham Standards Track [Page 15] + +RFC 8288 Web Linking October 2017 + + +6. Internationalisation Considerations + + Link targets may need to be converted to URIs in order to express + them in serialisations that do not support IRIs. This includes the + Link HTTP header field. + + Similarly, the anchor parameter of the Link header field does not + support IRIs; therefore, IRIs must be converted to URIs before + inclusion there. + + Relation types are defined as URIs, not IRIs, to aid in their + comparison. It is not expected that they will be displayed to end + users. + + Note that registered Relation Names are required to be lowercase + ASCII letters. + +7. References + +7.1. Normative References + + [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate + Requirement Levels", BCP 14, RFC 2119, + DOI 10.17487/RFC2119, March 1997, + . + + [RFC3864] Klyne, G., Nottingham, M., and J. Mogul, "Registration + Procedures for Message Header Fields", BCP 90, RFC 3864, + DOI 10.17487/RFC3864, September 2004, + . + + [RFC3986] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform + Resource Identifier (URI): Generic Syntax", STD 66, + RFC 3986, DOI 10.17487/RFC3986, January 2005, + . + + [RFC3987] Duerst, M. and M. Suignard, "Internationalized Resource + Identifiers (IRIs)", RFC 3987, DOI 10.17487/RFC3987, + January 2005, . + + [RFC5234] Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax + Specifications: ABNF", STD 68, RFC 5234, + DOI 10.17487/RFC5234, January 2008, + . + + [RFC5646] Phillips, A., Ed. and M. Davis, Ed., "Tags for Identifying + Languages", BCP 47, RFC 5646, DOI 10.17487/RFC5646, + September 2009, . + + + +Nottingham Standards Track [Page 16] + +RFC 8288 Web Linking October 2017 + + + [RFC6838] Freed, N., Klensin, J., and T. Hansen, "Media Type + Specifications and Registration Procedures", BCP 13, + RFC 6838, DOI 10.17487/RFC6838, January 2013, + . + + [RFC7230] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer + Protocol (HTTP/1.1): Message Syntax and Routing", + RFC 7230, DOI 10.17487/RFC7230, June 2014, + . + + [RFC7231] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer + Protocol (HTTP/1.1): Semantics and Content", RFC 7231, + DOI 10.17487/RFC7231, June 2014, + . + + [RFC8126] Cotton, M., Leiba, B., and T. Narten, "Guidelines for + Writing an IANA Considerations Section in RFCs", BCP 26, + RFC 8126, DOI 10.17487/RFC8126, June 2017, + . + + [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC + 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, + May 2017, . + + [RFC8187] Reschke, J., "Indicating Character Encoding and Language + for HTTP Header Field Parameters", RFC 8187, + DOI 10.17487/RFC8187, September 2017, + . + + [W3C.REC-css3-mediaqueries-20120619] + Rivoal, F., "Media Queries", W3C Recommendation + REC-css3-mediaqueries-20120619, June 2012, + . + +7.2. Informative References + + [RFC2046] Freed, N. and N. Borenstein, "Multipurpose Internet Mail + Extensions (MIME) Part Two: Media Types", RFC 2046, + DOI 10.17487/RFC2046, November 1996, + . + + [RFC2818] Rescorla, E., "HTTP Over TLS", RFC 2818, + DOI 10.17487/RFC2818, May 2000, + . + + + + + + +Nottingham Standards Track [Page 17] + +RFC 8288 Web Linking October 2017 + + + [RFC4287] Nottingham, M., Ed. and R. Sayre, Ed., "The Atom + Syndication Format", RFC 4287, DOI 10.17487/RFC4287, + December 2005, . + + [RFC6265] Barth, A., "HTTP State Management Mechanism", RFC 6265, + DOI 10.17487/RFC6265, April 2011, + . + + [W3C.REC-html5-20141028] + Hickson, I., Berjon, R., Faulkner, S., Leithead, T., + Navara, E., O'Connor, T., and S. Pfeiffer, "HTML5", W3C + Recommendation REC-html5-20141028, October 2014, + . + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +Nottingham Standards Track [Page 18] + +RFC 8288 Web Linking October 2017 + + +Appendix A. Notes on Other Link Serialisations + + Header fields (Section 3) are only one serialisation of links; other + specifications have defined alternative serialisations. + +A.1. Link Serialisation in HTML + + HTML motivated the original syntax of the Link header field, and many + of the design decisions in this document are driven by a desire to + stay compatible with it. + + In HTML, the link element can be mapped to links as specified here by + using the "href" attribute for the target URI, and "rel" to convey + the relation type, as in the Link header field. The context of the + link is the URI associated with the entire HTML document. HTML also + defines several attributes on links that can be seen as target + attributes, including "media", "hreflang", "type", and "sizes". + + Section 4.8 of HTML5 [W3C.REC-html5-20141028] defines modern HTML + links. That document links to the Microformats Wiki as a registry; + over time, the IANA registry ought to mirror its contents and, + ideally, eventually replace it (although that depends on the HTML + community). + + Surveys of existing HTML content have shown that unregistered link + relation types that are not URIs are (perhaps inevitably) common. + Consuming HTML implementations ought not consider such unregistered + short links to be errors, but rather relation types with a local + scope (i.e., their meaning is specific and perhaps private to that + document). + + Finally, the HTML specification gives a special meaning when the + "alternate" relation types coincide with other relation types in the + same link. Such links ought to be serialised in the Link header + field using a single list of relation-types (e.g., rel="alternate + stylesheet") to preserve this relationship. + +A.2. Link Serialisation in Atom + + Atom [RFC4287] is a link serialisation that conveys links in the + atom:link element, with the "href" attribute indicating the link + target and the "rel" attribute containing the relation type. The + context of the link is either a feed locator or an entry ID, + depending on where it appears; generally, feed-level links are + obvious candidates for transmission as a Link header field. + + When serialising an atom:link into a Link header field, it is + necessary to convert link targets (if used) to URIs. + + + +Nottingham Standards Track [Page 19] + +RFC 8288 Web Linking October 2017 + + + Atom defines extension relation types in terms of IRIs. This + specification redefines them as URIs, to simplify and reduce errors + in their comparison. + + Atom allows registered link relation types to be serialised as + absolute URIs using a prefix, "http://www.iana.org/assignments/ + relation/". This prefix is specific to the Atom serialisation. + + Furthermore, link relation types are always compared in a case- + sensitive fashion; therefore, registered link relation types SHOULD + be converted to their registered form (usually, lowercase) when + serialised in an Atom document. + + Note also that while the Link header field allows multiple relations + to be serialised in a single link, atom:link does not. In this case, + a single link-value may map to several atom:link elements. + + As with HTML, atom:link defines some attributes that are not + explicitly mirrored in the Link header field syntax, but they can + also be used as link-extensions to maintain fidelity. + +Appendix B. Algorithms for Parsing Link Header Fields + + This appendix outlines a set of non-normative algorithms: for parsing + the Link header(s) out of a header set, for parsing a Link header + field value, and algorithms for parsing generic parts of the field + value. + + These algorithms are more permissive than the ABNF defining the + syntax might suggest; the error handling embodied in them is a + reasonable approach, but not one that is required. As such they are + advisory only, and in cases where there is disagreement, the correct + behaviour is defined by the body of this specification. + +B.1. Parsing a Header Set for Links + + This algorithm can be used to parse the Link header fields that a + HTTP header set contains. Given a header_set of (string field_name, + string field_value) pairs, assuming ASCII encoding, it returns a list + of link objects. + + 1. Let field_values be a list containing the members of header_set + whose field_name is a case-insensitive match for "link". + + 2. Let links be an empty list. + + + + + + +Nottingham Standards Track [Page 20] + +RFC 8288 Web Linking October 2017 + + + 3. For each field_value in field_values: + 1. Let value_links be the result of Parsing a Link Field Value + (Appendix B.2) from field_value. + 2. Append each member of value_links to links. + + 4. Return links. + +B.2. Parsing a Link Field Value + + This algorithm parses zero or more comma-separated link-values from a + Link header field. Given a string field_value, assuming ASCII + encoding, it returns a list of link objects. + + 1. Let links be an empty list. + + 2. While field_value has content: + 1. Consume any leading OWS. + 2. If the first character is not "<", return links. + 3. Discard the first character ("<"). + 4. Consume up to but not including the first ">" character or + end of field_value and let the result be target_string. + 5. If the next character is not ">", return links. + 6. Discard the leading ">" character. + 7. Let link_parameters be the result of Parsing Parameters + (Appendix B.3) from field_value (consuming zero or more + characters of it). + 8. Let target_uri be the result of relatively resolving (as per + [RFC3986], Section 5.2) target_string. Note that any base + URI carried in the payload body is NOT used. + 9. Let relations_string be the second item of the first tuple + of link_parameters whose first item matches the string "rel" + or the empty string ("") if it is not present. + 10. Split relations_string on RWS (removing it in the process) + into a list of string relation_types. + 11. Let context_string be the second item of the first tuple of + link_parameters whose first item matches the string + "anchor". If it is not present, context_string is the URL + of the representation carrying the Link header [RFC7231], + Section 3.1.4.1, serialised as a URI. Where the URL is + anonymous, context_string is null. + 12. Let context_uri be the result of relatively resolving (as + per [RFC3986], Section 5.2) context_string, unless + context_string is null, in which case context is null. Note + that any base URI carried in the payload body is NOT used. + 13. Let target_attributes be an empty list. + + + + + + +Nottingham Standards Track [Page 21] + +RFC 8288 Web Linking October 2017 + + + 14. For each tuple (param_name, param_value) of link_parameters: + 1. If param_name matches "rel" or "anchor", skip this + tuple. + 2. If param_name matches "media", "title", "title*", or + "type" and target_attributes already contains a tuple + whose first element matches the value of param_name, + skip this tuple. + 3. Append (param_name, param_value) to target_attributes. + 15. Let star_param_names be the set of param_names in the + (param_name, param_value) tuples of link_parameters where + the last character of param_name is an asterisk ("*"). + 16. For each star_param_name in star_param_names: + 1. Let base_param_name be star_param_name with the last + character removed. + 2. If the implementation does not choose to support an + internationalised form of a parameter named + base_param_name for any reason (including, but not + limited to, it being prohibited by the parameter's + specification), remove all tuples from link_parameters + whose first member is star_param_name, and skip to the + next star_param_name. + 3. Remove all tuples from link_parameters whose first + member is base_param_name. + 4. Change the first member of all tuples in link_parameters + whose first member is star_param_name to + base_param_name. + 17. For each relation_type in relation_types: + 1. Case-normalise relation_type to lowercase. + 2. Append a link object to links with the target + target_uri, relation type of relation_type, context of + context_uri, and target attributes target_attributes. + + 3. Return links. + +B.3. Parsing Parameters + + This algorithm parses the parameters from a header field value. + Given input, an ASCII string, it returns a list of (string + parameter_name, string parameter_value) tuples that it contains. + input is modified to remove the parsed parameters. + + 1. Let parameters be an empty list. + + 2. While input has content: + 1. Consume any leading OWS. + 2. If the first character is not ";", return parameters. + 3. Discard the leading ";" character. + 4. Consume any leading OWS. + + + +Nottingham Standards Track [Page 22] + +RFC 8288 Web Linking October 2017 + + + 5. Consume up to but not including the first BWS, "=", ";", or + "," character, or up to the end of input, and let the result + be parameter_name. + 6. Consume any leading BWS. + 7. If the next character is "=": + 1. Discard the leading "=" character. + 2. Consume any leading BWS. + 3. If the next character is DQUOTE, let parameter_value be + the result of Parsing a Quoted String (Appendix B.4) + from input (consuming zero or more characters of it). + 4. Else, consume the contents up to but not including the + first ";" or "," character, or up to the end of input, + and let the results be parameter_value. + 5. If the last character of parameter_name is an asterisk + ("*"), decode parameter_value according to [RFC8187]. + Continue processing input if an unrecoverable error is + encountered. + 8. Else: + 1. Let parameter_value be an empty string. + 9. Case-normalise parameter_name to lowercase. + 10. Append (parameter_name, parameter_value) to parameters. + 11. Consume any leading OWS. + 12. If the next character is "," or the end of input, stop + processing input and return parameters. + +B.4. Parsing a Quoted String + + This algorithm parses a quoted string, as per [RFC7230], + Section 3.2.6. Given input, an ASCII string, it returns an unquoted + string. input is modified to remove the parsed string. + + 1. Let output be an empty string. + + 2. If the first character of input is not DQUOTE, return output. + + 3. Discard the first character. + + 4. While input has content: + 1. If the first character is a backslash ("\"): + 1. Discard the first character. + 2. If there is no more input, return output. + 3. Else, consume the first character and append it to + output. + 2. Else, if the first character is DQUOTE, discard it and return + output. + 3. Else, consume the first character and append it to output. + + 5. Return output. + + + +Nottingham Standards Track [Page 23] + +RFC 8288 Web Linking October 2017 + + +Appendix C. Changes from RFC 5988 + + This specification has the following differences from its + predecessor, RFC 5988: + + o The initial relation type registrations were removed, since + they've already been registered by RFC 5988. + o The introduction has been shortened. + o The "Link Relation Application Data" registry has been removed. + o Incorporated errata. + o Updated references. + o Link cardinality was clarified. + o Terminology was changed from "target IRI" and "context IRI" to + "link target" and "link context", respectively. + o Made assigning a URI to registered relation types serialisation + specific. + o Removed misleading statement that the Link header field is + semantically equivalent to HTML and Atom links. + o More carefully defined and used "link serialisations" and "link + applications." + o Clarified the cardinality of target attributes (generically and + for "type"). + o Corrected the default link context for the Link header field, to + be dependent upon the identity of the representation (as per + RFC 7231). + o Defined a suggested parsing algorithm for the Link header. + o The value space of target attributes and their definition has been + specified. + o The ABNF has been updated to be compatible with [RFC7230]. In + particular, whitespace is now explicit. + o Some parameters on the HTTP header field can now appear as a + token. + o Parameters on the HTTP header can now be valueless. + o Handling of quoted strings is now defined by [RFC7230]. + o The "type" header field parameter now needs to be quoted (as + "token" does not allow "/"). + +Author's Address + + Mark Nottingham + + Email: mnot@mnot.net + URI: https://www.mnot.net/ + + + + + + + + +Nottingham Standards Track [Page 24] + -- cgit v1.2.3