Open In App

unicode_literals in Python

Last Updated : 21 Jun, 2021
Improve
Improve
Like Article
Like
Save
Share
Report

Unicode is also called Universal Character set. ASCII uses 8 bits(1 byte) to represents a character and can have a maximum of 256 (2^8) distinct combinations. The issue with the ASCII is that it can only support the English language but what if we want to use another language like Hindi, Russian, Chinese, etc. We didn’t have enough space in ASCII to covers up all these languages and emojis. This is where Unicode comes, Unicode provides us a huge table to which can store ASCII table and also the extent to store other languages, symbols, and emojis.

We actually can not save the text as Unicode directly. Because Unicode is just an abstract representation of the text data. We need some kind of encoding/mapping to map each character to a certain number. If a character uses more than 1 byte(8-bits), then all that bytes need to be packed as a single unit (think of a box with more than one item). This boxing method is called the UTF-8 method. In UTF-8 character can occupy a minimum of 8 bits and in UTF-16 a character can occupy a minimum of 16-bits. UTF is just an algorithm that turns Unicode into bytes and read it back

Normally, in python2 all string literals are considered as byte strings by default but in the later version of python, all the string literals are Unicode strings by default. So to make all the strings literals Unicode in python we use the following import :

from __future__ import unicode_literals

If we are using an older version of python, we need to import the unicode_literals from the future package. This import will make python2 behave as python3 does. This will make the code cross-python version compatible.

Python 2

Python




import sys
 
# checking the default encoding of string
print "The default encoding for python2 is:",
sys.getdefaultencoding()


Output:

The default encoding for python2 is: ascii

As in python2, the default encoding is ASCII we need to switch the encoding to utf-8. 

Python




from __future__ import unicode_literals
 
# creating variables to holds
# the letters in python word.
p = "\u2119"
y = "\u01b4"
t = "\u2602"
h = "\u210c"
o = "\u00f8"
n = "\u1f24"
 
# printing Python
# encoding to utf-8 from ascii
print(p+y+t+h+o+n).encode("utf-8")


 
 

Output

ℙƴ☂ℌøἤ

Python3:

 

Python3




# In python3
# By default the encoding is "utf-8"
import sys
 
# printing the default encoding
print("The default encoding for python3 is:", sys.getdefaultencoding())
 
# to define string as unicode
# we need to prefix every string with u"...."
p = u"\u2119"
y = u"\u01b4"
t = u"\u2602"
h = u"\u210c"
o = u"\u00f8"
n = u"\u1f24"
 
# printing Python
print(p+y+t+h+o+n)


Output:

The default encoding for python3 is: utf-8
ℙƴ☂ℌøἤ

Here,

Sr. no. 

Unicode

Description

1.

U+2119

it will display double-struck capital P

2.

U+01B4

it will display the Latin small letter Y with a hook.

3.

U+2602

it will display an umbrella.

4.

U+210C

it will display the capital letter H.

5.

U+00F8

it will display the Latin small letter O with a stroke.

6.

U+1F24

it will display the Greek letter ETA.



Similar Reads

Important differences between Python 2.x and Python 3.x with examples
In this article, we will see some important differences between Python 2.x and Python 3.x with the help of some examples. Differences between Python 2.x and Python 3.x Here, we will see the differences in the following libraries and modules: Division operatorprint functionUnicodexrangeError Handling_future_ modulePython Division operatorIf we are p
5 min read
Python program to build flashcard using class in Python
In this article, we will see how to build a flashcard using class in python. A flashcard is a card having information on both sides, which can be used as an aid in memoization. Flashcards usually have a question on one side and an answer on the other. Particularly in this article, we are going to create flashcards that will be having a word and its
2 min read
Python | Merge Python key values to list
Sometimes, while working with Python, we might have a problem in which we need to get the values of dictionary from several dictionaries to be encapsulated into one dictionary. This type of problem can be common in domains in which we work with relational data like in web developments. Let's discuss certain ways in which this problem can be solved.
4 min read
Reading Python File-Like Objects from C | Python
Writing C extension code that consumes data from any Python file-like object (e.g., normal files, StringIO objects, etc.). read() method has to be repeatedly invoke to consume data on a file-like object and take steps to properly decode the resulting data. Given below is a C extension function that merely consumes all of the data on a file-like obj
3 min read
Python | Add Logging to a Python Script
In this article, we will learn how to have scripts and simple programs to write diagnostic information to log files. Code #1 : Using the logging module to add logging to a simple program import logging def main(): # Configure the logging system logging.basicConfig(filename ='app.log', level = logging.ERROR) # Variables (to make the calls that follo
2 min read
Python | Add Logging to Python Libraries
In this article, we will learn how to add a logging capability to a library, but don’t want it to interfere with programs that don’t use logging. For libraries that want to perform logging, create a dedicated logger object, and initially configure it as shown in the code below - Code #1 : C/C++ Code # abc.py import logging log = logging.getLogger(_
2 min read
JavaScript vs Python : Can Python Overtop JavaScript by 2020?
This is the Clash of the Titans!! And no...I am not talking about the Hollywood movie (don’t bother watching it...it's horrible!). I am talking about JavaScript and Python, two of the most popular programming languages in existence today. JavaScript is currently the most commonly used programming language (and has been for quite some time!) but now
5 min read
Python | Visualizing O(n) using Python
Introduction Algorithm complexity can be a difficult concept to grasp, even presented with compelling mathematical arguments. This article presents a tiny Python program that shows the relative complexity of several typical functions. It can be easily adapted to other functions. Complexity. Why it matters? Computational complexity is a venerable su
3 min read
Python | Index of Non-Zero elements in Python list
Sometimes, while working with python list, we can have a problem in which we need to find positions of all the integers other than 0. This can have application in day-day programming or competitive programming. Let's discuss a shorthand by which we can perform this particular task. Method : Using enumerate() + list comprehension This method can be
6 min read
Python | Convert list to Python array
Sometimes while working in Python we can have a problem in which we need to restrict the data elements to just one type. A list can be heterogeneous, can have data of multiple data types and it is sometimes undesirable. There is a need to convert this to a data structure that restricts the type of data. Convert List to Array PythonBelow are the met
3 min read
Practice Tags :