Reading a CSV file is the first task in nearly every data pipeline, yet the options have multiplied enough in 2026 that picking the wrong one costs you either a dependency or your machine's RAM. Three tools cover every realistic case: the standard-library csv module, pandas, and polars. Knowing which to reach for takes about two minutes.
What changed in 2026
- Polars hit 1.x stability and is now a first-class choice; many teams that used pandas for performance have migrated their ingestion layer.
- pandas 3.x dropped the implicit copy warning and overhauled nullable dtypes — type inference is more predictable than it was in 2.x.
- Python's csv module is unchanged and that is a feature; it remains a zero-dependency, fast enough option for simple pipelines.
- Arrow-backed DataFrames are the default interchange format; both pandas and polars can export to Arrow without copying.
Option 1 — csv.DictReader (no dependencies)
Best for: shell scripts, Lambda functions, cases where you can't install pandas.
import csv
with open("sales.csv", encoding="utf-8-sig") as fh:
reader = csv.DictReader(fh)
for row in reader:
print(row["amount"], row["date"])
DictReader maps each row to a dict keyed by column name. utf-8-sig silently strips the BOM that Excel inserts — skip it and you get date as your first column name.
Option 2 — pandas read_csv (most common)
Best for: analysis, transformation, joining with other DataFrames.
import pandas as pd
df = pd.read_csv(
"sales.csv",
parse_dates=["date"],
dtype={"amount": "float64"},
encoding="utf-8-sig",
)
print(df.head())
print(df.dtypes)
Key parameters worth knowing:
| Parameter |
What it does |
parse_dates |
Parse listed columns as datetime |
dtype |
Override inferred types (prevents surprises) |
usecols |
Load only named columns (cuts memory) |
chunksize |
Return an iterator of DataFrames instead of one big frame |
na_values |
Extra strings to treat as NaN (e.g. "N/A", "-") |
sep |
Override delimiter (e.g. sep=";" for European CSVs) |
Reading a large CSV in chunks
chunks = pd.read_csv("big.csv", chunksize=100_000)
result = pd.concat(
[chunk[chunk["amount"] > 0] for chunk in chunks],
ignore_index=True,
)
Option 3 — polars (fast, memory-efficient)
Best for: files over ~50 MB, multi-core machines, pipelines that need speed.
import polars as pl
# Eager — reads the whole file
df = pl.read_csv("sales.csv", try_parse_dates=True)
# Lazy — scans without loading; only materialises after .collect()
df = (
pl.scan_csv("sales.csv")
.filter(pl.col("amount") > 0)
.select(["date", "amount", "region"])
.collect()
)
print(df)
scan_csv pushes column selection and row filters down before any data is loaded — on a 1 GB file the difference vs pandas is usually 3–5×.
What changed in 2026
| Library |
Best case |
Typical 100 MB read |
Memory overhead |
| csv.DictReader |
tiny files |
~8 s |
near zero |
| pandas 3.x |
analysis |
~1.8 s |
~3–5× file size |
| polars 1.x |
large files |
~0.4 s |
~1.5–2× file size |
How to pick
- No extra packages allowed? →
csv.DictReader.
- Doing analysis / joining / groupby? → pandas.
- File over 50 MB or you care about wall-clock speed? → polars.
- Unsure of schema? → pandas with
low_memory=False first; tighten dtypes after.
- File from Excel? → always
encoding="utf-8-sig" regardless of library.
Common mistakes
Forgetting encoding="utf-8-sig". Files exported from Excel carry a BOM. The result is a first-column name that starts with — hard to debug and never caught by the code itself.
Loading all columns when you only need three. usecols in pandas and select in a polars lazy frame cut memory significantly; use them from the start.
Using dtype=object everywhere. Leaving every column as object (string) defeats type inference and makes filtering slow. Define dtypes explicitly for the columns you know.
Assuming the delimiter is always a comma. CSVs from European locales often use ;. Check before assuming; pd.read_csv("f.csv", sep=None, engine="python") will detect it automatically.
What to skip
- Reading a CSV into SQLite then querying it — skip the round-trip; filter in polars or pandas directly.
- **Manually splitting lines with
str.split(",")**. Quoted fields with commas inside will break you.
- xlrd for CSV — xlrd is for
.xls files; CSV is plain text.
FAQ
What if my CSV has no header row?
Pass header=None to pandas and has_header=False to polars; then assign df.columns = ["col1", "col2", ...] afterward.
How do I handle mixed date formats in a column?
In pandas: pd.to_datetime(df["date"], format="mixed") (added in 2.0). In polars: pl.col("date").str.to_datetime(format="%Y-%m-%d", strict=False).
Can I read a CSV directly from a URL?
Yes in pandas: pd.read_csv("https://example.com/data.csv"). In polars you need to fetch first: import httpx; pl.read_csv(httpx.get(url).content).
What about semicolons, tabs, or pipes as delimiters?
Set sep=";", sep="\t", or sep="|" in pandas. Polars uses the same parameter name: separator=";".
Where to go next