C++ Tokens

In C++, tokens can be defined as the smallest building block of C++ programs that the compiler understands. Every word in a C++ source code can be considered a token.

Types of Tokens in C++

We have several types of tokens each of which serves a specific purpose in the syntax and semantics of C++. Below are the main types of tokens in C++:

1. Identifiers
2. Keywords
3. Constants
4. Strings
5. Special Symbols
6. Operators

1. Identifiers

In C++, entities like variables, functions, classes, or structs must be given unique names within the program so that they can be uniquely identified. The unique names given to these entities are known as identifiers.

It is recommended to choose valid and relevant names of identifiers to write readable and maintainable programs. Keywords cannot be used as an identifier because they are reserved words to do specific tasks. In the below example, “first_name’ is an identifier.

string first_name = "Raju";

We have to follow a set of rules to define the name of identifiers as follows:

1. An identifier can only begin with a letter or an underscore(_).
2. An identifier can consist of letters (A-Z or a-z), digits (0-9), and underscores (_). White spaces and Special characters can not be used as the name of an identifier.
3. Keywords cannot be used as an identifier because they are reserved words to do specific tasks. For example, string, int, class, struct, etc.
4. Identifier must be unique in its namespace.
5. As C++ is a case-sensitive language so identifiers such as ‘first_name’ and ‘First_name’ are different entities.

Here are some examples of valid and invalid identifiers in C++:

Valid Identifiers	Invalid Identifiers
_name	#name (Cannot start with special character except ‘_’)
Number89	2num (Cannot start with a digit)
first_name	first name (Cannot include space)
_last_name_	string (Cannot be same as a keyword)

2. Keywords

Keywords in C++ are the tokens that are the reserved words in programming languages that have their specific meaning and functionalities within a program. Keywords cannot be used as an identifier to name any variables.

For example, a variable or function cannot be named as ‘int’ because it is reserved for declaring integer data type.

There are 95 keywords reserved in C++. Some of the main Keywords are:

break	try	catch	char	class	const	continue
default	delete	auto	else	friend	for	float
long	new	operator	private	protected	public	return
short	sizeof	static	this	typedef	enum	throw
mutable	struct	case	register	switch	and	or
namespace	static_cast	goto	not	xor	bool	do
double	int	unsigned	void	virtual	union	while

3. Constants

Constants are the tokens in C++ that are used to define variables at the time of initialization and the assigned value cannot be changed after that.

We can define the constants in C++ in two ways that are using the ‘const’ keyword and ‘#define’ preprocessor directive. Let’s see both methods one by one.

1. Define Constants using the ‘const’ Keyword in C++

Constants are the values that do not change during the execution of the program once defined. We can make constant any type of data such as integer, float, string, and characters. A literal is a constant value assigned to a constant variable.

Syntax

const data_type variable_name = value;

2. Define Constants using the ‘#define’ preprocessor directive in C++

#define preprocessor can be used to define constant identifiers and the identifier is replaced with the defined value throughout the program where ever it is used. It is defined globally outside the main function.

Syntax

// The constant_Name is replaced by its value throughout the program where ever it is used
#define constant_Name value

4. Strings

In C++, a string is not a built-in data type like ‘int’, ‘char’, or ‘float’. It is a class available in the STL library which provides the functionality to work with a sequence of characters, that represents a string of text.

When we define any variable using the ‘string’ keyword we are actually defining an object that represents a sequence of characters. We can perform various methods on the string provided by the string class such as length(), push_backk(), and pop_back().

Syntax of declaring a string

string variable_name;

Initialize the string object with string within the double inverted commas (“).

string variable_name = "This is string";

5. Special Symbols

Special symbols are the token characters having specific meanings within the syntax of the programming language. These symbols are used in a variety of functions, including ending the statements, defining control statements, separating items, and more.

Below are the most common special symbols used in C++ programming:

• Semicolon (;): It is used to terminate the statement.
• Square brackets []: They are used to store array elements.
• Curly Braces {}: They are used to define blocks of code.
• Scope resolution (::): Scope resolution operator is used to access members of namespaces, classes, etc.
• Dot (.): Dot operator also called member access operator used to access class and struct members.
• Assignment operator ‘=’: This operator is used to assign values to variables.
• Double-quote (“): It is used to enclose string literals.
• Single-quote (‘): It is used to enclose character literals.

6. Operators

C++ operators are special symbols that are used to perform operations on operands such as variables, constants, or expressions. A wide range of operators is available in C++ to perform a specific type of operations which includes arithmetic operations, comparison operations, logical operations, and more.

For example (A+B), in which ‘A’ and ‘B’ are operands, and ‘+’ is an arithmetic operator which is used to add two operands.

Types of Operators

1. Unary Operators
2. Binary Operators
3. Ternary Operators

1. Unary Operators

Unary operators are used with single operands only. They perform the operations on a single variable. For example, increment and decrement operators.

• Increment operator ( ++ ): It is used to increment the value of an operand by 1.
• Decrement operator ( — ): It is used to decrement the value of an operand by 1.

2. Binary Operators

They are used with the two operands and they perform the operations between the two variables. For example (A<B), less than (<) operator compares A and B, returns true if A is less than B else returns false.

• Arithmetic Operators: These operators perform basic arithmetic operations on operands. They include ‘+’, ‘-‘, ‘*’, ‘/’, and ‘%’
• Comparison Operators: These operators are used to compare two operands and they include ‘==’, ‘!=’, ‘<‘, ‘>’, ‘<=’, and ‘>=’.
• Logical Operators: These operators perform logical operations on boolean values. They include ‘&&’, ‘||’, and ‘!‘.
• Assignment Operators: These operators are used to assign values to variables and they include ‘variables‘, ‘-=‘, ‘*=‘, ‘/=‘, and ‘%=’.
• Bitwise Operators: These operators perform bitwise operations on integers. They include‘&’, ‘|’, ‘^’, ‘~’, ‘<<‘, and ‘>>‘.

3. Ternary Operator

The ternary operator is the only operator that takes three operands. It is also known as a conditional operator that is used for conditional expressions.

Syntax:

Expression1 ? Expression2 : Expression3;

If ‘Expression1’ became true then ‘Expression2’ will be executed otherwise ‘Expression3’ will be executed.