Comprehensive Guide to Pandas: The Ultimate Cheat Sheet

Pandas is an open-source data manipulation and analysis library built on top of Python. It provides easy-to-use data structures like DataFrame and Series that facilitate data handling for all kinds of data analysis tasks. It is widely used for handling structured data, data cleaning, and preparation, which is a crucial step in data science workflows. Whether it's time series data, heterogeneous data, or data that comes in CSV, Excel, SQL databases, or JSON format, Pandas offers powerful tools to make working with this data much easier.

1. Importing Pandas

Before using any Pandas functionality, you need to import the library. It is commonly imported as pd to keep the syntax concise.

import pandas as pd

2. Pandas Data Structures

Series

A Series is a one-dimensional labeled array, capable of holding any data type (integer, string, float, etc.). It can be created from a list, NumPy array, or a dictionary.

# Create a Pandas Series from a list
s = pd.Series([1, 2, 3, 4])

Expected Output:

0    1
1    2
2    3
3    4
dtype: int64

DataFrame

A DataFrame is a two-dimensional labeled data structure, similar to a table in a database or an Excel spreadsheet. It consists of rows and columns. Each column can have a different data type.

# Create a DataFrame from a dictionary
data = {'Name': ['Alice', 'Bob', 'Charlie'], 'Age': [24, 27, 22], 'City': ['New York', 'London', 'Berlin']}
df = pd.DataFrame(data)

Expected Output:

      Name  Age      City
0    Alice   24  New York
1      Bob   27    London
2  Charlie   22    Berlin

3. Creating DataFrames and Series

From a Dictionary

data = {'col1': [1, 2, 3], 'col2': [4, 5, 6]}
df = pd.DataFrame(data)

From a List of Lists

data = [[1, 2, 3], [4, 5, 6]]
df = pd.DataFrame(data, columns=["A", "B", "C"])

Expected Output:

   A  B  C
0  1  2  3
1  4  5  6

4. Inspecting DataFrames

Pandas provides several methods to inspect and get information about your data.

• df.head(n) – Returns the first n rows.
• df.tail(n) – Returns the last n rows.
• df.info() – Provides summary information about the DataFrame.
• df.describe() – Generates descriptive statistics of the DataFrame.

# Inspecting the DataFrame
print(df.head())
print(df.tail())
print(df.info())
print(df.describe())

Expected Output:

   A  B  C
0  1  2  3
1  4  5  6

   A  B  C
0  1  2  3
1  4  5  6

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 2 entries, 0 to 1
Data columns (total 3 columns):
 #   Column  Non-Null Count  Dtype
---  ------  --------------  -----
 0   A       2 non-null      int64
 1   B       2 non-null      int64
 2   C       2 non-null      int64
dtypes: int64(3)
memory usage: 128.0 bytes

       A    B    C
count  2.0  2.0  2.0
mean   2.5  3.5  4.5
std    2.1  2.1  2.1
min    1.0  2.0  3.0
25%    1.5  2.5  3.5
50%    2.0  3.0  4.0
75%    2.5  3.5  4.5
max    4.0  5.0  6.0

5. Indexing, Slicing, and Subsetting Data

Accessing Columns

You can access columns either using dot notation or by indexing with square brackets.

# Dot notation
print(df.A)

# Bracket notation
print(df["B"])

Accessing Rows by Index

You can use .iloc[] for integer-location based indexing and .loc[] for label-based indexing.

# Using iloc (index-based)
print(df.iloc[0])  # Access first row

# Using loc (label-based)
print(df.loc[0])  # Access first row using label

Slicing Data

You can slice DataFrames to get subsets of data. You can slice rows or columns.

# Select specific rows and columns
subset = df.loc[0:1, ["A", "C"]]

Expected Output:

   A  C
0  1  3
1  4  6

6. Modifying DataFrames

Adding Columns

You can add columns directly to the DataFrame by assigning values.

df['D'] = [7, 8]  # Adding a new column

Modifying Column Values

You can modify the values of a column by accessing it and assigning new values.

df['A'] = df['A'] * 2  # Modify the 'A' column

Dropping Columns or Rows

You can drop rows or columns using the drop() function.

df = df.drop(columns=['D'])  # Dropping a column
df = df.drop(index=1)  # Dropping a row by index

7. Handling Missing Data

Handling missing data is a critical task. Pandas provides several functions to handle missing data.

• df.isnull() – Detects missing values (returns a DataFrame of booleans).
• df.notnull() – Detects non-missing values (returns a DataFrame of booleans).
• df.fillna(value) – Fills missing values with a specified value.
• df.dropna() – Removes rows with missing values.

df = df.fillna(0)  # Fill missing data with 0
df = df.dropna()  # Drop rows with any missing values

8. Data Aggregation and Grouping

GroupBy

The groupby() function is used for splitting the data into groups, applying a function, and then combining the results.

# Grouping by a column and calculating the sum
grouped = df.groupby('City').sum()

Aggregation Functions

You can apply various aggregation functions like sum(), mean(), min(), max(), etc.

# Aggregating data using mean
df.groupby('City').mean()

9. Sorting and Ranking

Sorting Data

You can sort a DataFrame by one or more columns using the sort_values() function.

# Sorting by a column in ascending order
df_sorted = df.sort_values(by='Age')

# Sorting by multiple columns
df_sorted = df.sort_values(by=['Age', 'Name'], ascending=[True, False])

Ranking

You can rank the values in a DataFrame using rank().

df['Rank'] = df['Age'].rank()

10. Merging, Joining, and Concatenating DataFrames

Merging DataFrames

You can merge two DataFrames based on a common column or index.

df1 = pd.DataFrame({'A': ['A0', 'A1', 'A2'], 'B': ['B0', 'B1', 'B2']})
df2 = pd.DataFrame({'A': ['A0', 'A1', 'A2'], 'C': ['C0', 'C1', 'C2']})
merged_df = pd.merge(df1, df2, on='A')

Concatenating DataFrames

You can concatenate DataFrames along rows or columns using concat().

df1 = pd.DataFrame([[1, 2], [3, 4]], columns=['A', 'B'])
df2 = pd.DataFrame([[5, 6], [7, 8]], columns=['A', 'B'])
concat_df = pd.concat([df1, df2], axis=0)

Conclusion

Pandas is a versatile tool for data manipulation, from importing and cleaning data to performing complex operations. This cheat sheet provides a quick overview of some of the most common Pandas features, helping you make your data analysis workflow more efficient.