Pandas 3.1 Explained: Inplace Deprecation, Select, Filter and Faster CSV Reads

Mostly nothing breaks, and that is the trap. pandas 3.1 starts warning about code you have written a hundred times, like df.dropna(inplace=True). One change to dates can also alter numbers without any warning at all.

Picture a common setup. A small online shop runs a Python script every night. It reads the day's orders from a CSV file, drops orders with no rating, sorts them and sums sales by city. The setup is made up, but every output below was run on pandas 3.1.0.

Here is the data we will use for the whole guide. It has six orders, two of them without a rating. The placed_at column stores time as a plain number: seconds since 1 January 1970, called a Unix timestamp.

import pandas as pd
from io import StringIO

csv_text = """order_id,city,amount,qty,placed_at,rating
1001,Pune,1250,2,1792650600,4.5
1002,Delhi,480,1,1792654200,
1003,Pune,3200,4,1792661400,3.0
1004,Chennai,999,1,1792668600,5.0
1005,Delhi,2150,3,1792675800,4.0
1006,Kochi,640,2,1792683000,
"""
orders = pd.read_csv(StringIO(csv_text))
print(pd.__version__)   # 3.1.0
print(orders)

Run this after pip install pandas==3.1.0 and you should see the same six rows. A DataFrame is pandas' table type, and orders is one. Orders 1002 and 1006 show NaN, which means not a number, pandas' marker for a missing value.

You need five words. Each one comes up again below, so it is worth a minute now.

• Method: a function that belongs to a table, called with a dot, like orders.sort_values("amount").
• inplace=True: an option that asked a method to change your table directly instead of handing back a new one.
• Deprecation: a notice that a feature still works today but will be removed in a later version.
• Warning: a message Python prints without stopping your program. Python can also hide some warnings, which matters later.
• Copy-on-Write: since pandas 3.0, a table you derive from another behaves like a separate copy. pandas only copies the data when one of them is changed.

pandas 3.1 adds two ways to pick data, speeds up CSV reading, deprecates inplace on most methods and changes the default time unit for numbers turned into dates. The table lists what each change means for the nightly orders script.

Two names in the table need a word first. Pandas4Warning is the warning pandas prints for anything that will change or disappear in pandas 4.0. A mask is a column of True and False values, one per row, that says which rows to keep.

Because on these methods it no longer saves any work. For methods like dropna and sort_values, the pandas plan for this change, called PDEP-8, describes the keyword as essentially just syntactic sugar for reassigning the new result. Under the hood, sort_values(inplace=True) computes a new result and loads it back into your existing table object.

Copy-on-Write made the shorthand unnecessary. PDEP-8 explains that under Copy-on-Write, methods return a light copy that shares data until something changes. So orders = orders.sort_values("amount") behaves like the inplace version, minus the confusion.

Think of a variable as a name tag. Your table is a box of order slips, and orders is the tag on it. sort_values fills a new box with the slips in order. Reassigning moves your tag to the new box.

inplace=True instead emptied your old box and refilled it with the sorted slips. Anyone else holding a tag on that box saw the change too, which matters for functions later. Where the picture breaks: pandas does not always copy every slip, because Copy-on-Write lets two boxes share slips until one changes.

Which methods lose inplace and which keep it

The split follows one question: can the method change values inside the existing table? Methods that change the table's shape, order or labels lose the keyword. Methods that only change values keep it.

Two details surprised us in testing. Passing inplace=False also warns, because pandas objects to the keyword itself. And fillna(..., inplace=True) returns the table itself instead of None, a change that arrived in pandas 3.0.

# Old style: still runs in 3.1.0, but warns
orders.sort_values("amount", inplace=True)

# New style: keep the result by assigning it back
orders = orders.sort_values("amount")
print(orders["order_id"].tolist())
# [1002, 1006, 1004, 1001, 1005, 1003]

Run the first line as a script and Python prints a Pandas4Warning. The message says the keyword in DataFrame.sort_values will be removed in a future version, and names PDEP-8. The second line runs silently.

Replace each call with an assignment, but read the surrounding code first. The plain swap is safe at the top level of a script. Inside a function, the same swap can quietly change what your program does.

1. 1

   Find every inplace call

   Search your project for inplace=. Check each method against the table above. Index methods such as rename and set_names keep inplace without a warning.

2. 2

   Leave the value methods alone

   fillna, replace, where, mask, ffill, bfill, clip and interpolate still accept inplace. You may still switch them for consistency.

3. 3

   Swap top-level calls for assignment

   Turn orders.dropna(inplace=True) into orders = orders.dropna(). Delete inplace=False outright on these methods, since passing it at all now warns.

4. 4

   Fix calls inside functions

   Make the function return the new table, and assign it where the function is called. The next example shows why.

5. 5

   Chain the steps if you like

   Without inplace, each method returns a table, so several steps can sit in one readable chain.

6. 6

   Check with warnings switched on

   Run python -W default::DeprecationWarning to list every deprecated call that runs. In CI, the automatic checks on every code change, use -W error::DeprecationWarning. It stops at the first call, which blocks new ones.

The function-argument trap

Pause and predict: three helper functions each try to drop unrated orders. How many rows does each caller end up with? The table starts with six rows, and two have no rating.

def drop_unrated_old(df):
    df.dropna(subset=["rating"], inplace=True)   # changes the caller's frame

def drop_unrated_wrong(df):
    df = df.dropna(subset=["rating"])            # only changes a local name

def drop_unrated(df):
    return df.dropna(subset=["rating"])          # return the new frame

a = orders.copy(); drop_unrated_old(a)
b = orders.copy(); drop_unrated_wrong(b)
c = drop_unrated(orders)
print(len(a), len(b), len(c))

What just happened? It prints 4 6 4. The old version changed the caller's table through inplace, so it ends with 4 rows. The naive rewrite only moved a local name tag inside the function, so the caller still has 6. Returning the new table and assigning it fixes it.

Chaining the cleaning steps

top = (
    orders
    .dropna(subset=["rating"])
    .sort_values("amount", ascending=False)
    .reset_index(drop=True)
)
print(top[["order_id", "city", "amount"]])
#    order_id     city  amount
# 0      1003     Pune    3200
# 1      1005    Delhi    2150
# 2      1001     Pune    1250
# 3      1004  Chennai     999

Each line returns a new table that feeds the next one. The bracket around the chain lets you put one step per line, which makes code reviews easier.

pd.col("amount") is a placeholder that means the amount column of whatever table it is used on. It arrived in pandas 3.0, and 3.1 lets more methods accept it. select returns a new table with only the columns you name, plus any new columns you compute. Plain names pick existing columns. Keyword arguments create new columns, and the keyword becomes the column name.

summary = orders.select(
    "order_id",
    "city",
    per_item=pd.col("amount") / pd.col("qty"),
)
print(summary)
#    order_id     city    per_item
# 0      1001     Pune  625.000000
# 1      1002    Delhi  480.000000
# 2      1003     Pune  800.000000
# 3      1004  Chennai  999.000000
# 4      1005    Delhi  716.666667
# 5      1006    Kochi  320.000000

Here per_item is computed from two columns using pd.col. Later keywords can use columns made earlier in the same call, just like assign, the older method that adds columns. Unlike assign, which keeps every column, select keeps only what you list.

• orders.select("city") returns a one-column table. orders["city"] returns a Series, a single column, which behaves differently.
• A missing name raises KeyError, so a typo fails loudly instead of returning an empty table.
• An expression that combines columns, like pd.col("amount") / pd.col("qty"), has no name. Passed by position, it raises TypeError, so name it with a keyword or add .rename("per_item").

filter now keeps the rows where a mask is True, and treats a missing mask value as False by default. Before 3.1, filter only picked rows or columns by their labels. A pd.col expression or a function can be passed without writing cond=.

big = orders.filter(pd.col("amount") > 1000)
print(big[["order_id", "city", "amount"]])
#    order_id   city  amount
# 0      1001   Pune    1250
# 2      1003   Pune    3200
# 4      1005  Delhi    2150

Missing values need care. In our orders table, a missing rating is NaN, and comparing NaN with 4 simply gives False, so the row is dropped. convert_dtypes switches to nullable types, which store a missing value as NA, meaning unknown. With NA, filter lets you choose what happens.

rated = orders.convert_dtypes()   # nullable types: blanks become NA
good = pd.col("rating") >= 4

print(rated.filter(good)["order_id"].tolist())
# [1001, 1004, 1005]               missing ratings dropped
print(rated.filter(good, na=True)["order_id"].tolist())
# [1001, 1002, 1004, 1005, 1006]   missing ratings kept
rated.filter(good, na="raise")
# ValueError: The mask contains missing values

Here the two missing ratings are NA. The default drops them, na=True keeps them and na="raise" stops with an error. On the original float column, na=True would not bring them back, because the mask holds False there, not NA.

Numbers turned into dates or durations now default to microsecond precision. In 3.0.6, pd.to_datetime(..., unit="s") kept the unit you passed, here seconds. The release notes list this under backwards-incompatible changes, and it prints no warning.

placed = pd.to_datetime(orders["placed_at"], unit="s")
print(placed.dtype)                              # datetime64[us]  (3.0.6: datetime64[s])
print(placed.iloc[0])                            # 2026-10-22 06:30:00
print(placed.astype("int64").iloc[0])            # 1792650600000000  (3.0.6: 1792650600)
print(placed.dt.as_unit("s").astype("int64").iloc[0])   # 1792650600 on both

The date itself is the same. What changes is any code that turns it back into a number. A nightly script that wrote astype("int64") into a database would now write values a million times larger. Call .dt.as_unit("s") first if you need seconds on both versions.

The scope is narrow. It applies to to_datetime, to_timedelta, Timestamp and Timedelta built from whole numbers with a unit. Whole-valued decimals switch too, such as a Unix-time column with blanks, which read_csv stores as float. A value with a fraction, like 1792650600.5, gives nanoseconds. Parsing date strings gave microseconds already in 3.0.6.

Because Pandas4Warning is a kind of DeprecationWarning, which Python hides by default. Python shows it only when the code that triggers it is your main script. An inplace call inside an imported module, like a helpers file, prints nothing.

We checked both cases on 3.1.0. A script calling sort_values(inplace=True) printed the warning. The same call inside an imported helper module ran silently. This follows Python's default warning filter, not anything special in pandas.

The PDEP-8 plan said the warning would start as a DeprecationWarning and become a FutureWarning, which Python always shows, later in the 3.x series. In 3.1.0 it is still the hidden kind. The 3.0.0 release notes set the policy: these warnings switch to FutureWarning in the last 3.x minor release. Which release that is had not been announced as of 25 October 2026, so pin your version.

# 1. Confirm the version
python -c "import pandas; print(pandas.__version__)"
# 3.1.0

# 2. Turn every deprecation warning into an error for one run
python -W error::DeprecationWarning daily_report.py
# ... (traceback trimmed)
# pandas.errors.Pandas4Warning: The inplace keyword in DataFrame.dropna
# is deprecated and will be removed in a future version (PDEP-8).
# See https://pandas.pydata.org/docs/dev/whatsnew/v3.1.0.html#deprecation-inplace for more details.

# 3. List every warning that fires, without stopping
python -W default::DeprecationWarning daily_report.py

# 4. Block new ones in a test suite
python -m pytest -W error::DeprecationWarning

Expect a version line of 3.1.0, then a traceback, the error report Python prints, whose last lines name Pandas4Warning and the method, followed by a link to the release notes. If another library's deprecation fires first, the run stops there instead. Use -W default::DeprecationWarning to list them all without stopping.

pytest lists these warnings in its warnings summary (we used pytest 8.4.2), so read that summary after upgrading. One trap: python -W error::pandas.errors.Pandas4Warning does not work. Python rejects the dotted name with an "invalid module name" message and ignores the option. Use the DeprecationWarning form, or call warnings.filterwarnings("error", category=pd.errors.Pandas4Warning) in code.

Yes, in our test it was about 3.3 times faster, even on one thread. The release notes say a single-threaded read is typically two to three times faster than in 3.0. On machines with more than one CPU, large local files are also read on several threads by default.

We made a CSV of 2,000,000 made-up orders with seven columns, 100,284,182 bytes in total. Each setting ran 21 times, in turns, and the table gives the median. The file was already in memory, so disk speed is not included.

Read the 3.1.0 column with care. On this two-core virtual machine, the two-thread read was not reliably faster than one thread. The fastest single runs dropped to about 0.2 seconds, but the medians stayed within noise. A machine with more free cores may gain more; we did not measure one.

Notice the last row too. On this file, pandas' own C parser in 3.1.0 beat the pyarrow engine, an optional parser built on the PyArrow library. That is one file on one machine, so time your own files before switching.

When does read_csv use several threads?

Only when the read is simple enough to split safely. We read the rules in the 3.1.0 source and checked them with a counter on the parallel code path.

The default thread count is the smaller of your CPU count and 6, further limited on Linux by the CPUs your process may use. The user guide says containers with a CPU limit, such as under Docker or Kubernetes, get fewer threads. Set mode.max_threads to 1 when your program already runs work in parallel.

import pandas as pd

print(pd.get_option("mode.max_threads"))   # None: pandas decides

with pd.option_context("mode.max_threads", 1):
    df = pd.read_csv("orders_big.csv")     # forced serial read

The documentation says the result is always identical to a serial read. We compared both on our 2,000,000-row file and on a file whose pincode column switched from numbers to text near the end. Both matched exactly, including the same mixed-type warning.

Two details are worth knowing. The release notes say parallel reads need at least 5 MB. In the code it is really 5 MiB, 5,242,880 bytes: a 5,000,016-byte file read serially and a 5,300,032-byte file read in parallel. Passing dtype= does not block parallel reading.

File engines are the next most common. An engine is the library pandas calls to read or write a file format. Each of these printed a Pandas4Warning on 3.1.0 in our tests.

• Parquet: engine="fastparquet" and engine="auto" are deprecated. The release notes say the fastparquet library has been retired. Use pyarrow, the default engine, built on the PyArrow library.
• Old Excel .xls files: the xlrd engine is deprecated, and a plain read_excel on a .xls file still uses it and warns. Pass engine="calamine", which needs the python-calamine package.
• Modern .xlsx files: if python-calamine is installed and you pass no engine, pandas warns that the default will change from openpyxl to calamine. Pass engine= explicitly to choose.
• Dates: dayofweek, dayofyear and daysinmonth are deprecated on Timestamp and on a column's .dt accessor, and so is .dt.weekday. Use day_of_week, day_of_year and days_in_month. Timestamp.weekday() is unchanged.

The same release changes a few other defaults that rarely touch beginner code. Examples are read-only arrays from Index.values and read_csv inferring numbers inside category columns. Read the release notes if your code uses those.

Treat it as a dependency upgrade question. Interviewers want to hear how you find risk, test it and roll it out, not a list of features.

1. 1

   Name what changed

   Say that pandas 3.1 deprecates inplace on most methods, adds select and filter masks, speeds up read_csv and changes a date default.

2. 2

   Split loud from silent

   Deprecations warn, but the date unit change does not. Silent changes need tests on real outputs, not just a clean log.

3. 3

   Make warnings visible

   Explain that these are DeprecationWarnings, hidden in imported modules, so CI runs with -W error::DeprecationWarning.

4. 4

   Measure, do not assume

   Time read_csv on production-sized files and real core counts. Cap mode.max_threads where a service already runs work in parallel.

5. 5

   Roll out in stages

   Pin pandas, upgrade one job, compare outputs row by row with the old version, then upgrade the rest.