Pandas is still the default tool for tabular data manipulation in Python, despite Polars catching up fast. For most data engineering and ML preprocessing work in 2026, you will need pandas fluency. The 3.x series changes enough that old tutorials will teach you patterns that now silently fail or produce warnings. Here is what actually matters in 2026.
What changed in 2026
- Copy-on-Write (CoW) is enabled by default in pandas 3. Chained assignment (
df["col"][mask] = value) no longer modifies the original — it silently no-ops or raises. The fix is df.loc[mask, "col"] = value.
- Arrow backend is stable and default-optional.
pd.options.mode.dtype_backend = "pyarrow" cuts memory 2–5x and speeds up string operations dramatically.
- pandas 3 dropped Python 3.9 support — target 3.11+ for best performance with the new backends.
- Polars interop improved — you can round-trip DataFrames between pandas and Polars via Arrow zero-copy in a single line.
The learning order
- Series and DataFrame creation — from dicts, from CSV, from lists.
- Indexing: iloc, loc, boolean masks — the single most-used skill in real work.
- Basic cleaning —
dropna, fillna, astype, rename, drop.
- Groupby and aggregation —
groupby().agg(), transform, apply (sparingly).
- Merge and join — inner, left, right, cross; understanding key alignment.
- Reshaping —
pivot_table, melt, stack, unstack.
- Time series —
to_datetime, resample, dt accessor.
- Performance — vectorized ops, Arrow backend, chunked reading with
chunksize.
Copy-on-Write — the most important 2026 change
import pandas as pd
df = pd.DataFrame({"score": [10, 20, 30]})
# WRONG in pandas 3 — chained indexing; may silently no-op
df["score"][0] = 99 # SettingWithCopyWarning → now silent no-op
# CORRECT — use loc
df.loc[0, "score"] = 99 # works reliably
# CORRECT — for conditional updates
mask = df["score"] > 15
df.loc[mask, "score"] = df.loc[mask, "score"] * 2
Indexing: the thing to master first
# iloc — position-based
first_row = df.iloc[0]
top_five = df.iloc[:5]
cell = df.iloc[2, 1] # row 2, column index 1
# loc — label-based (includes both ends)
subset = df.loc[df["score"] > 20] # boolean mask
df.loc[mask, "label"] = "high" # conditional assignment
Groupby and aggregation
# The pattern you will use daily
result = (
df.groupby("category")
.agg(
mean_score=("score", "mean"),
count=("score", "count"),
max_score=("score", "max"),
)
.reset_index()
)
Arrow backend for large data
# Enable at session start for 2–5x memory reduction
pd.options.mode.dtype_backend = "pyarrow"
# Or per-read
df = pd.read_csv("large.csv", dtype_backend="pyarrow")
print(df.dtypes) # shows ArrowDtype instead of numpy types
print(df.memory_usage(deep=True).sum() / 1e6, "MB")
pandas vs Polars in 2026
| Factor |
pandas 3 |
Polars |
| Ecosystem (sklearn, statsmodels) |
Excellent |
Growing |
| Familiar to most data scientists |
Yes |
No |
| Performance on large data |
Good with Arrow |
Excellent |
| Lazy evaluation |
No |
Yes |
| SQL-style expressions |
Limited |
Full |
| Learning curve for pandas users |
Near-zero |
Moderate |
| Best for |
Industry standard, ML prep |
Performance-critical ETL |
How to pick your learning approach
- Work with your own data — find a CSV you care about and explore it. Real questions drive faster learning than toy datasets.
- Read the pandas 3 migration guide — 30 minutes reading it saves 10 hours of debugging CoW surprises.
- Practice in Jupyter — pandas + Jupyter is the canonical environment;
.head(), .info(), .describe() as first calls on any dataset.
- Learn merges by trying to break them — join a dataset on a key that has duplicates; understand why the row count explodes.
- Profile before optimizing — use
%timeit in Jupyter to measure before rewriting with Arrow or chunking.
Common mistakes
Row-by-row iteration. for i, row in df.iterrows() is 100–1000x slower than vectorized operations. Use df["new"] = df["a"] + df["b"] or np.where.
Reading entire large files into memory. Use pd.read_csv(..., chunksize=10_000) or switch to Polars/DuckDB for files over ~500MB.
Forgetting reset_index() after groupby. The resulting DataFrame has the groupby keys as the index, which causes alignment issues downstream.
Using apply as a loop shortcut. df.apply(lambda row: ..., axis=1) is just a slow loop in disguise. Find the vectorized equivalent.
Ignoring dtypes. Reading a column as object instead of category for low-cardinality strings wastes memory and slows groupby. Check dtypes after every read.
What to skip
- pandas 1.x / 2.x idioms —
inplace=True still works but is being deprecated; avoid it.
DataFrame.append() — removed in pandas 2; use pd.concat instead.- Heavy
apply chains — switch to Polars or DuckDB if your data transformation is too slow for vectorized pandas.
FAQ
Should I learn pandas or Polars first in 2026?
Learn pandas first. The ecosystem, job market, and library integrations (scikit-learn, statsmodels, matplotlib) all assume pandas DataFrames. Add Polars later for performance-critical pipelines.
How big of a dataset can pandas handle?
Rule of thumb: comfortably up to ~5x your available RAM with Arrow backend. For 100M+ row datasets, reach for DuckDB or Polars with lazy evaluation.
What is the fastest way to speed up existing pandas code?
First, enable the Arrow backend. Second, replace loops with vectorized ops. Third, check if a groupby+agg replaces a manual loop. These three changes cover 90% of real slowdowns.
Is pandas good for time-series data?
Yes — the DatetimeIndex, resample, rolling, and dt accessor make it strong for time-series. For very large financial tick data, consider Arctic or TimescaleDB for storage.
Where to go next