The Selection Toolkit
`.loc` (By Label)
The standard for selecting data by its **index and column names** (labels). It's explicit, reliable, and your primary tool for both getting and setting data.
`.iloc` (By Position)
Selects data by its **integer position**, starting from 0. Use this when the order of your data is more important than its labels.
`.query()` (By String)
Filters rows using a human-readable string expression. It's exceptionally clear for complex conditions but cannot be used to set values.
Quick Comparison: Selection Methods
| Feature | `.loc` | `.iloc` | `.query()` |
|---|---|---|---|
| Input Type | Labels & Booleans | Integers & Booleans | String expression |
| Can Set Values? | Yes | Yes | No |
| Best For | Reliable, explicit code | Positional logic | Readable, complex filters |
The Copy Conundrum: View vs. Copy
Understanding the difference between a view and a copy is the most important step in avoiding the dreaded `SettingWithCopyWarning`.
View
A "view" is a direct window into the original DataFrame's data. No new object is created. Modifying a view will change the original data and should be avoided.
Shallow Copy (`.copy()`)
Creates a new DataFrame object and copies the top-level data. However, any nested objects (like lists inside cells) are still referenced, not duplicated.
Deep Copy (`copy.deepcopy()`)
Creates a new DataFrame and recursively duplicates everything, including all nested objects. This guarantees complete independence but is slower.
Quick Comparison: View vs. Copy
| Feature | View | Shallow Copy (`.copy()`) | Deep Copy (`copy.deepcopy()`) |
|---|---|---|---|
| Independence | No | Partial (top-level data only) | Yes (fully independent) |
| Use Case | Internal pandas operations. Avoid creating these intentionally. | Standard practice for safely modifying a filtered DataFrame. | Rarely needed; only for DataFrames with complex nested objects. |
Pitfall: The Chained Indexing Trap
What is Chained Indexing?
It's when you use multiple indexing brackets back-to-back, like df['col'][row]. While it might work for selecting data, it's highly unreliable for assigning new values.
Why is it Dangerous for Assignment?
Pandas can't guarantee whether the first operation (`df['col']`) returns a **view** or a **copy**. If it returns a copy, your second operation (`[row] = value`) modifies that temporary object, which is then discarded. The original DataFrame remains unchanged, often with a SettingWithCopyWarning.
❌ Unsafe & Unreliable
✅ Safe & Guaranteed
The Definitive Guide to Brackets
Parentheses `()`
Used for **actions**: calling functions and methods, or controlling the order of operations.
Square Brackets `[]`
Used for **accessing**: selecting columns, filtering rows, or getting items from a list.
Curly Braces `{}`
Used for **creating**: building dictionaries or creating sets of unique items.
Quick Comparison: Bracket Usage
| Bracket | Mental Model | Primary Use Case |
|---|---|---|
| () | "Do something" | Calling a function or method (e.g., `df.head()`). |
| [] | "Get something" | Selecting columns or filtering rows (e.g., `df['Age']`). |
| {} | "Create something" | Creating a dictionary, often to build a DataFrame. |
Pitfall: The Tilde `~` Trap
Bitwise vs. Logical NOT
The tilde `~` operator in Python is a bitwise NOT, designed for integer arithmetic, not a logical `not` for boolean logic. This distinction is crucial and can lead to very confusing bugs if misused.
Why it's Confusing
In Python, `True` is treated as the integer `1`. The bitwise NOT of `1` (`~1`) is `-2`, not `False`.
Unexpected Behavior
Using `~` on a single boolean value.
Correct Approach
Using the `not` keyword.
The Golden Rule of Negation
Use the not keyword for single boolean values. Use the tilde ~ operator only for element-wise inversion of a pandas Series (e.g., df[~mask]).