Content      ..      1       2         ..

Working Draft, Standard for Programming Language C++ (N4713, 2017 year) - page 1

5.5

Alternative tokens

[lex.digraph]

1

Alternative token representations are provided for some operators and punctuators.¹²

2

In all respects of the language, each alternative token behaves the same, respectively, as its primary token,

except for its spelling.¹³ The set of alternative tokens is defined in Table 1.

Table 1 — Alternative tokens

Alternative Primary

Alternative Primary

Alternative Primary

<%

{

and

&&

and_eq

&=

%>

}

bitor

|

or_eq

|=

<:

[

or

||

xor_eq

^=

:>

]

xor

^

not

!

%:

#

compl

~

not_eq

!=

%:%:

##

bitand

&

5.6

Tokens

[lex.token]

token:

identifier

keyword

literal

operator

punctuator

1

There are five kinds of tokens: identifiers, keywords, literals,¹⁴ operators, and other separators. Blanks,

horizontal and vertical tabs, newlines, formfeeds, and comments (collectively, “white space”), as described

below, are ignored except as they serve to separate tokens. [ Note: Some white space is required to separate

otherwise adjacent identifiers, keywords, numeric literals, and alternative tokens containing alphabetic

characters.

— end note ]

5.7

Comments

[lex.comment]

1

The characters /* start a comment, which terminates with the characters */. These comments do not nest.

The characters // start a comment, which terminates immediately before the next new-line character. If

there is a form-feed or a vertical-tab character in such a comment, only white-space characters shall appear

between it and the new-line that terminates the comment; no diagnostic is required. [ Note: The comment

characters //, /*, and */ have no special meaning within a // comment and are treated just like other

characters. Similarly, the comment characters // and /* have no special meaning within a /* comment.

— end note ]

5.8

Header names

[lex.header]

header-name:

< h-char-sequence >

" q-char-sequence "

h-char-sequence:

h-char

h-char-sequence h-char

h-char:

any member of the source character set except new-line and >

q-char-sequence:

q-char

q-char-sequence q-char

q-char:

any member of the source character set except new-line and "

12) These include “digraphs” and additional reserved words. The term “digraph” (token consisting of two characters) is not

perfectly descriptive, since one of the alternative preprocessing-tokens is %:%: and of course several primary tokens contain two

characters. Nonetheless, those alternative tokens that aren’t lexical keywords are colloquially known as “digraphs”.

13) Thus the “stringized” values (19.3.2) of [ and <: will be different, maintaining the source spelling, but the tokens can

otherwise be freely interchanged.

14) Literals include strings and character and numeric literals.

§ 5.8

12

6

A character literal that begins with the letter L, such as L’z’, is a wide-character literal. A wide-character

literal has type wchar_t.¹⁹ The value of a wide-character literal containing a single c-char has value equal

to the numerical value of the encoding of the c-char in the execution wide-character set, unless the c-char

has no representation in the execution wide-character set, in which case the value is implementation-defined.

[Note: The type wchar_t is able to represent all members of the execution wide-character set (see 6.7.1).

— end note ] The value of a wide-character literal containing multiple c-char s is implementation-defined.

7

Certain non-graphic characters, the single quote ’, the double quote ", the question mark ?,²⁰ and the

backslash \, can be represented according to Table 8. The double quote " and the question mark ?, can

be represented as themselves or by the escape sequences \" and \? respectively, but the single quote ’

and the backslash \ shall be represented by the escape sequences \’ and \\ respectively. Escape sequences

in which the character following the backslash is not listed in Table 8 are conditionally-supported, with

implementation-defined semantics. An escape sequence specifies a single character.

Table 8 — Escape sequences

new-line

NL(LF)

\n

horizontal tab

HT

\t

vertical tab

VT

\v

backspace

BS

\b

carriage return

CR

\r

form feed

FF

\f

alert

BEL

\a

backslash

\

\\

question mark

?

\?

single quote

’

\’

double quote

"

\"

octal number

ooo

\ooo

hex number

hhh

\xhhh

8

The escape \ooo consists of the backslash followed by one, two, or three octal digits that are taken to specify

the value of the desired character. The escape \xhhh consists of the backslash followed by x followed by one

or more hexadecimal digits that are taken to specify the value of the desired character. There is no limit to

the number of digits in a hexadecimal sequence. A sequence of octal or hexadecimal digits is terminated by

the first character that is not an octal digit or a hexadecimal digit, respectively. The value of a character

literal is implementation-defined if it falls outside of the implementation-defined range defined for char (for

character literals with no prefix) or wchar_t (for character literals prefixed by L). [ Note: If the value of a

character literal prefixed by u, u8, or U is outside the range defined for its type, the program is ill-formed.

— end note ]

9

A universal-character-name is translated to the encoding, in the appropriate execution character set, of the

character named. If there is no such encoding, the universal-character-name is translated to an implementation-

defined encoding. [ Note: In translation phase 1, a universal-character-name is introduced whenever an actual

extended character is encountered in the source text. Therefore, all extended characters are described in

terms of universal-character-names. However, the actual compiler implementation may use its own native

character set, so long as the same results are obtained.

— end note ]

5.13.4

Floating literals

[lex.fcon]

floating-literal:

decimal-floating-literal

hexadecimal-floating-literal

decimal-floating-literal:

fractional-constant exponent-partopt floating-suffixopt

digit-sequence exponent-part floating-suffixopt

hexadecimal-floating-literal:

hexadecimal-prefix hexadecimal-fractional-constant binary-exponent-part floating-suffixopt

hexadecimal-prefix hexadecimal-digit-sequence binary-exponent-part floating-suffixopt

19) They are intended for character sets where a character does not fit into a single byte.

20) Using an escape sequence for a question mark is supported for compatibility with ISO C++ 2014 and ISO C.

§ 5.13.4

18

13

In translation phase 6 (5.2), adjacent string-literals are concatenated. If both string-literals have the same

encoding-prefix, the resulting concatenated string literal has that encoding-prefix. If one string-literal has

no encoding-prefix, it is treated as a string-literal of the same encoding-prefix as the other operand. If a

UTF-8 string literal token is adjacent to a wide string literal token, the program is ill-formed. Any other

concatenations are conditionally-supported with implementation-defined behavior. [ Note: This concatenation

is an interpretation, not a conversion. Because the interpretation happens in translation phase 6 (after

each character from a string literal has been translated into a value from the appropriate character set), a

string-literal’s initial rawness has no effect on the interpretation or well-formedness of the concatenation.

— end note ] Table 9 has some examples of valid concatenations.

Table 9 — String literal concatenations

Source

Means

Source

Means

Source

Means

u"a" u"b" u"ab"

U"a" U"b" U"ab"

L"a" L"b" L"ab"

u"a" "b" u"ab"

U"a" "b" U"ab"

L"a" "b" L"ab"

"a" u"b" u"ab"

"a" U"b" U"ab"

"a" L"b" L"ab"

Characters in concatenated strings are kept distinct.

[ Example:

"\xA" "B"

contains the two characters ’\xA’ and ’B’ after concatenation (and not the single hexadecimal character

’\xAB’). — end example ]

14

After any necessary concatenation, in translation phase 7 (5.2), ’\0’ is appended to every string literal so

that programs that scan a string can find its end.

15

Escape sequences and universal-character-names in non-raw string literals have the same meaning as in

character literals (5.13.3), except that the single quote ’ is representable either by itself or by the escape

sequence \’, and the double quote " shall be preceded by a \, and except that a universal-character-name in

a char16_t string literal may yield a surrogate pair. In a narrow string literal, a universal-character-name

may map to more than one char element due to multibyte encoding. The size of a char32_t or wide string

literal is the total number of escape sequences, universal-character-names, and other characters, plus one

for the terminating U’\0’ or L’\0’. The size of a char16_t string literal is the total number of escape

sequences, universal-character-names, and other characters, plus one for each character requiring a surrogate

pair, plus one for the terminating u’\0’.

[Note: The size of a char16_t string literal is the number of

code units, not the number of characters.

— end note ] Within char32_t and char16_t string literals, any

universal-character-names shall be within the range 0x0 to 0x10FFFF. The size of a narrow string literal is

the total number of escape sequences and other characters, plus at least one for the multibyte encoding of

each universal-character-name, plus one for the terminating ’\0’.

16

Evaluating a string-literal results in a string literal object with static storage duration, initialized from

the given characters as specified above. Whether all string literals are distinct (that is, are stored in

nonoverlapping objects) and whether successive evaluations of a string-literal yield the same or a different

object is unspecified. [ Note: The effect of attempting to modify a string literal is undefined.

— end note ]

5.13.6

Boolean literals

[lex.bool]

boolean-literal:

false

true

1

The Boolean literals are the keywords false and true. Such literals are prvalues and have type bool.

5.13.7

Pointer literals

[lex.nullptr]

pointer-literal:

nullptr

1

The pointer literal is the keyword nullptr. It is a prvalue of type std::nullptr_t. [ Note: std::nullptr_t

is a distinct type that is neither a pointer type nor a pointer-to-member type; rather, a prvalue of this type

is a null pointer constant and can be converted to a null pointer value or null member pointer value. See 7.11

and 7.12.

— end note ]

§ 5.13.7

21

Content      ..      1       2         ..

///////////////////////////////////////