Working with Binary Data in Python
Alright, lets get this out of the way! The basics are pretty standard:
- There are 8 bits in a byte
- Bits either consist of a 0 or a 1
- A byte can be interpreted in different ways, like binary octal or hexadecimal
Note: These are not character encodings, those come later. This is just a way to look at a set of 1’s and 0’s and see it in three different ways(or number systems).
Input : 10011011 Output : 1001 1011 ---- 9B (in hex) 1001 1011 ---- 155 (in decimal) 1001 1011 ---- 233 (in octal)
This clearly shows a string of bits can be interpreted differently in different ways. We often use the hex representation of a byte instead of the binary one because it is shorter to write, this is just a representation and not an interpretation.
Now that we know what a byte is and what it looks like, let us see how it is interpreted, mainly in strings. Character Encodings are a way to assign values to bytes or sets of bytes that represent a certain character in that scheme. Some encodings are ASCII(probably the oldest), Latin, and UTF-8(most widely used as of today. In a sense encodings are a way for computers to represent, send and interpret human readable characters. This means that a sentence in one encoding might become completely incomprehensible in another encoding.
Python and Bytes
From a developer’s point of view, the largest change in Python 3 is the handling of strings. In Python 2, the str type was used for two different kinds of values – text and bytes, whereas in Python 3, these are separate and incompatible types. This means that before Python3 we could treat a set of bytes as a string and work from there, this is not the case now, now we have a separate data type, called bytes. This data type can be briefly explained as a string of bytes, which essentially means, once the bytes data type is initialized it is immutable.
b'abc' 97 Traceback (most recent call last): File "bytesExample.py", line 4, in bytestr = 97 TypeError: 'bytes' object does not support item assignment
A bytestring is what it says it is simply a string of bytes, for example ‘© ? ?’ in ‘utf-8’ is
This presents another problem, we need to know the encoding of a binary string, because the same string in another encoding(latin-1) looks different.
Â© ð â
As seen above it is possible to encode or decode strings and binary strings using the encode() or decode() function. We need the encoding because in some encodings it is not possible to to decode the strings. This problem compounds when not using non Latin characters like Hebrew, Japanese and Chinese. Because in those languages more than one byte is assigned to each letter. But what do we use when we need to modify a set of bytes, we use a bytearray.
In Python, bitwise operators are used to perform bitwise calculations on integers. The integers are first converted into binary and then operations are performed on bit by bit, hence the name bitwise operators. The standard bitwise operations are demonstrated below.
Note: For more information, refer to Python Bitwise Operators
-241 0 255 165 1 30 0 1
Some Other Applications
Binary data provides several applications like we can check if the two files are similar or not using the binary data, we can also check for a whether a file is jpeg or not (or any other image format). Let’s see the below examples for better understanding.
Example 1: Checking if the two files are same or not. Here two text files are used with the data as follows –
Files do not match.
Example 2: Checking if the given image is jpeg or not.
Attention geek! Strengthen your foundations with the Python Programming Foundation Course and learn the basics.
To begin with, your interview preparations Enhance your Data Structures concepts with the Python DS Course. And to begin with your Machine Learning Journey, join the Machine Learning – Basic Level Course