Python - Time Series and Date functionality

Time Series and Date Functionality in Python

Introduction

Time series data is one of the most common forms of data encountered in data science, finance, and business analytics. A time series is a series of data points indexed in time order, typically equally spaced in intervals like days, months, or years. Python, through the Pandas library, provides robust support for working with time series and date/time functionality.

In this document, we explore time series analysis capabilities in Python, focusing on:

• Basic date/time handling
• DatetimeIndex and conversion
• Creating and manipulating time series
• Resampling, frequency conversion
• Time series slicing and indexing
• Shifting, lagging, and window operations
• Working with time zones
• Real-world use cases and best practices

Working with Python datetime Module

Creating Date and Time Objects

from datetime import datetime, date, time

dt = datetime(2023, 5, 15, 12, 30)
d = date(2023, 5, 15)
t = time(12, 30)

print(dt)
print(d)
print(t)

Current Date and Time

now = datetime.now()
today = date.today()

print(now)
print(today)

Datetime Arithmetic

from datetime import timedelta

dt1 = datetime(2023, 5, 15)
dt2 = dt1 + timedelta(days=10)
print(dt2)

Time Series Basics with Pandas

Creating Time Series

import pandas as pd

dates = pd.date_range(start='2023-01-01', periods=5, freq='D')
values = [100, 102, 98, 105, 110]

ts = pd.Series(values, index=dates)
print(ts)

DatetimeIndex

Pandas automatically creates a DatetimeIndex if the index is a date range.

print(ts.index)

Accessing Time Series Values

print(ts['2023-01-03'])
print(ts['2023-01'])

Converting to Datetime

Using to_datetime()

df = pd.DataFrame({
    'date': ['2023-01-01', '2023-01-02', '2023-01-03'],
    'value': [10, 20, 30]
})

df['date'] = pd.to_datetime(df['date'])
print(df.dtypes)

Datetime Format Parsing

df['date'] = pd.to_datetime(df['date'], format='%Y-%m-%d')

Date Ranges and Frequencies

Creating Ranges

date_range = pd.date_range(start='2023-01-01', end='2023-01-10', freq='D')
print(date_range)

Common Frequency Codes

• 'D' - Daily
• 'B' - Business Day
• 'H' - Hourly
• 'T' - Minutely
• 'S' - Secondly
• 'M' - Month End
• 'MS' - Month Start
• 'Q' - Quarter End

Time Series Indexing and Selection

Indexing by Date

print(ts['2023-01-01'])

Slicing Time Series

print(ts['2023-01-01':'2023-01-03'])

Boolean Masking

print(ts[ts > 100])

Using loc with Timestamps

print(ts.loc['2023-01-04'])

Resampling and Frequency Conversion

Resample to Different Frequency

monthly = ts.resample('M').mean()
print(monthly)

Downsampling

Converting from high to low frequency (e.g., daily to monthly).

downsampled = ts.resample('2D').sum()
print(downsampled)

Upsampling

Converting from low to high frequency.

upsampled = ts.resample('H').ffill()
print(upsampled.head(10))

Rolling and Window Functions

Rolling Mean

rolling = ts.rolling(window=2).mean()
print(rolling)

Expanding Window

expanding = ts.expanding().mean()
print(expanding)

Applying Custom Function

custom = ts.rolling(window=3).apply(lambda x: max(x) - min(x))
print(custom)

Time Series Shifting and Lagging

Shifting Values

shifted = ts.shift(1)
print(shifted)

Lag and Change Calculation

diff = ts - ts.shift(1)
print(diff)

Lead Operation

lead = ts.shift(-1)
print(lead)

Working with Time Zones

Localizing Time Series

localized = ts.tz_localize('UTC')
print(localized)

Converting to Another Time Zone

converted = localized.tz_convert('Asia/Kolkata')
print(converted)

Datetime with Time Zone

dt_with_tz = pd.to_datetime('2023-01-01 10:00').tz_localize('US/Eastern')
print(dt_with_tz)

Period and PeriodIndex

Creating Periods

period = pd.Period('2023-01', freq='M')
print(period)

PeriodIndex Example

pindex = pd.period_range(start='2023-01', periods=4, freq='Q')
print(pindex)

Period-Based Time Series

values = [100, 120, 130, 140]
ts_period = pd.Series(values, index=pindex)
print(ts_period)

Timestamp and Period Conversion

To Timestamp

ts_from_period = ts_period.to_timestamp()
print(ts_from_period)

To Period

ts_to_period = ts.to_period('M')
print(ts_to_period)

Handling Missing Data in Time Series

Creating Missing Dates

ts_missing = ts.reindex(pd.date_range('2023-01-01', '2023-01-10'))
print(ts_missing)

Filling Missing Values

filled = ts_missing.ffill()
print(filled)

Interpolating Time Series

interpolated = ts_missing.interpolate()
print(interpolated)

Time Series Visualization (Optional)

Plotting Time Series

import matplotlib.pyplot as plt

ts.plot(title='Sample Time Series')
plt.xlabel('Date')
plt.ylabel('Value')
plt.show()

Real-World Use Case Examples

1. Financial Time Series

stock_data = pd.DataFrame({
    'date': pd.date_range(start='2023-01-01', periods=5, freq='B'),
    'price': [100, 102, 105, 107, 110]
})
stock_data.set_index('date', inplace=True)
returns = stock_data['price'].pct_change()
print(returns)

2. Web Traffic Analysis

traffic = pd.Series([200, 240, 250, 300, 280], 
                    index=pd.date_range('2023-03-01', periods=5, freq='D'))

print(traffic.rolling(window=3).mean())

3. IoT Sensor Data

iot_data = pd.Series([25.0, 25.5, 26.1, 26.4, 26.9],
                     index=pd.date_range('2023-06-01 00:00', periods=5, freq='H'))

print(iot_data.resample('2H').mean())

Best Practices

• Always convert date strings to datetime using to_datetime()
• Use date_range() for generating consistent time intervals
• Use resample() for rescaling frequencies
• Use shift() and rolling() for lag analysis and smoothing
• Work with Periods for calendar-aware analysis
• Visualize data to spot trends and anomalies

Time series analysis is fundamental in data science. Whether you are forecasting sales, tracking web traffic, or analyzing stock prices, mastering Python’s time and date functionalities is essential. With Pandas, Python provides a full set of tools to handle timestamps, periods, resampling, and time zone-aware data. By understanding time series indexing, resampling, rolling statistics, and missing data handling, you will be well-equipped to manage and analyze temporal data with confidence.