Python | Pandas dataframe.shift()

Python is a great language for doing data analysis, primarily because of the fantastic ecosystem of data-centric Python packages. Pandas is one of those packages and makes importing and analyzing data much easier. Pandas dataframe.shift() function Shift index by the desired number of periods with an optional time freq. This function takes a scalar parameter called the period, which represents the number of shifts to be made over the desired axis. This function is very helpful when dealing with time-series data.

Pandas Dataframe.shift() Syntax

Syntax:DataFrame.shift(periods=1, freq=None, axis=0) 

Parameters : 
periods : Number of periods to move, can be positive or negative 
freq : DateOffset, timedelta, or time rule string, optional Increment to use from the tseries module or time rule (e.g. ‘EOM’). See Notes 
axis : {0 or ‘index’, 1 or ‘columns’}

Return : shifted : DataFrame

What dataframe.shift() Function in Pandas?

The `DataFrame.shift()` function in Pandas is a method that shifts the values of a DataFrame along a specified axis. It allows for repositioning data in either a forward or backward direction by a specified number of positions. This operation is useful for creating time-lagged features in time series data or comparing current values with past or future values in a DataFrame. The `periods` parameter determines the number of positions to shift, with positive values shifting the data down and negative values shifting it up along the specified axis.

Pandas Dataframe.shift() Examples

Below is an explanation of commonly used methods with examples for the Pandas `shift()` function.

1. Python shift() Function in Pandas
2. Shift Columns in DataFrame using Shift()
3. Shift Index Axis in Negative Direction
4. Shift Time-Series Column Axis by 2 Periods
5. Shift Column Axis in Positive Direction
6. Shift in Negative Direction by Some Periods

Python shift() Function in Pandas

In this example, the below code uses Pandas to create a DataFrame with a datetime index, where the rows are indexed in 12-hour intervals starting from ’01/01/2000′. The DataFrame has four columns (A, B, C, D) with corresponding numerical values in each cell.

Python3

# importing pandas as pd
import pandas as pd

# Creating row index values for our data frame
ind = pd.date_range('01 / 01 / 2000', periods=5, freq='12H')

# Creating a dataframe with 4 columns
# using "ind" as the index for our dataframe
df = pd.DataFrame({"A": [1, 2, 3, 4, 5],
                   "B": [10, 20, 30, 40, 50],
                   "C": [11, 22, 33, 44, 55],
                   "D": [12, 24, 51, 36, 2]},
                  index=ind)
print(df)

Output:

                      A   B   C   D
2000-01-01 00:00:00   1  10  11  12
2000-01-01 12:00:00   2  20  22  24
2000-01-02 00:00:00   3  30  33  51
2000-01-02 12:00:00   4  40  44  36
2000-01-03 00:00:00   5  50  55   2

 Shift Columns in DataFrame using Shift()

Let’s use the dataframe.shift() function to shift the index axis by 2 periods in the positive direction 

Python3

# shift index axis by two periods in positive direction
# axis = 0 is set by default
df.shift(2, axis=0)

Output:

                       A   B   C   D
2000-01-01 00:00:00 NaN NaN NaN NaN
2000-01-01 12:00:00 NaN NaN NaN NaN
2000-01-02 00:00:00   1  10  11  12
2000-01-02 12:00:00   2  20  22  24
2000-01-03 00:00:00   3  30  33  51

Shift Index Axis in Negative Direction

Let’s shift the index axis in the negative direction by some periods

Python3

# shift index axis by two periods in negative direction
# axis = 0 is set by default
df.shift(-2, axis=0)

Output : 

                       A     B     C     D
2000-01-01 00:00:00  3.0  30.0  33.0  51.0
2000-01-01 12:00:00  4.0  40.0  44.0  36.0
2000-01-02 00:00:00  5.0  50.0  55.0   2.0
2000-01-02 12:00:00  NaN   NaN   NaN   NaN
2000-01-03 00:00:00  NaN   NaN   NaN   NaN

Shift Time-Series Column Axis by 2 Periods

In below example the code utilizes Pandas to construct a DataFrame with a datetime index, spanning five rows at 12-hour intervals from ’01/01/2000′. The DataFrame contains four columns (A, B, C, D) with corresponding numerical values, and it is printed at the end.

Python3

# importing pandas as pd
import pandas as pd

# Creating row index values for our data frame
# We have taken time frequency to be of 12 hours interval
# We are generating five index value using "period = 5" parameter

ind = pd.date_range('01 / 01 / 2000', periods=5, freq='12H')

# Creating a dataframe with 4 columns
# using "ind" as the index for our dataframe
df = pd.DataFrame({"A": [1, 2, 3, 4, 5],
                   "B": [10, 20, 30, 40, 50],
                   "C": [11, 22, 33, 44, 55],
                   "D": [12, 24, 51, 36, 2]},
                  index=ind)

# Print the dataframe
df

Output :

                     A   B   C   D
2000-01-01 00:00:00  1  10  11  12
2000-01-01 12:00:00  2  20  22  24
2000-01-02 00:00:00  3  30  33  51
2000-01-02 12:00:00  4  40  44  36
2000-01-03 00:00:00  5  50  55   2

Shift Column Axis in Positive Direction

Let’s use the dataframe.shift() function to shift the column axis by 2 periods in a positive direction

Python3

# shift column axis by two periods in positive direction
df.shift(2, axis=1)

Output:

                       A     B     C     D
2000-01-01 00:00:00 NaN   1.0  10.0  11.0
2000-01-01 12:00:00 NaN   2.0  20.0  22.0
2000-01-02 00:00:00 NaN   3.0  30.0  33.0
2000-01-02 12:00:00 NaN   4.0  40.0  44.0
2000-01-03 00:00:00 NaN   5.0  50.0  55.0

Shift in Negative Direction by Some Periods

Let’s shift the column axis in the negative direction by some periods

Python3

# shift column axis by two periods in negative direction
df.shift(-2, axis=1)

   A     B     C     D
2000-01-01 00:00:00  1.0  2.0  3.0  4.0
2000-01-01 12:00:00  20.0  30.0  40.0  50.0
2000-01-02 00:00:00  22.0  33.0  44.0  51.0
2000-01-02 12:00:00  36.0   2.0   NaN   NaN
2000-01-03 00:00:00   NaN   NaN   NaN   NaN