Session 3 · pandas core - learn python data analysis with phoebe

Part 0

The DataFrame is home base

Session 2 gave you the NumPy array - one typed block of numbers. A pandas DataFrame is what you get when you give those arrays names, an index, and the ability to sit side by side in a table. It is where every remaining session lives: cleaning, grouping, joining, charting, and modeling all happen on the DataFrame. Get fluent moving around it and everything downstream gets easier. Everything today builds directly on the boolean masks and dtypes you just learned - pandas simply wraps them in labels.

Live - presented in session Self-study - read after class ★ Build-along demo The project: the data job market

★ What you walk out with today The confidence to select any rows or columns you want, filter the LinkedIn postings with real conditions (salary above a threshold, remote roles, title contains "data"), and compute quick summaries - value_counts, describe, nunique - to answer "what are the top job titles and hiring locations?"

Part 1 · covers Kaggle Pandas L1 + McKinney ch5

Series and DataFrame 5 min live

Two structures, and the second is just many of the first stacked together. A Series is one labeled column: an index plus its values. A DataFrame is a dict of Series that all share one index - which is why columns line up row-for-row even after you filter or sort them.

🔍 Click to zoom - a DataFrame's anatomy, and a single column pulled out as a Series

LiveSeries: a labeled array2 min▶

A Series is a NumPy array (from Session 2) with an index bolted on. The values still have a single dtype and still support vectorized math and boolean masks - you've just gained labels, so you can ask for an element by name, not only by position.

★ Make a Seriesimport pandas as pd pay = pd.Series([95000, 160000, None], index=["analyst", "ml_eng", "ds"]) pay["ml_eng"] # 160000.0 - fetch by label pay.mean() # skips the missing value automatically pay > 100000 # boolean mask - same as NumPy, now labeled

Real world

When you run jobs["title"] on the LinkedIn data, you get a Series of 124,000 titles with the row index preserved. That preserved index is what lets you filter to "remote roles" and still line those rows up against their salaries and companies later - nothing drifts out of order.

Self-studyDataFrame: aligned columns2 min read▶

You'll rarely build a DataFrame by hand - read_csv does it for you - but seeing the dict-of-Series construction demystifies it. Every column shares the same index, so pandas can align them automatically even after reordering.

★ DataFrame from a dict of columnsdf = pd.DataFrame({ "title": ["Data Analyst", "ML Engineer", "Data Scientist"], "location": ["New York, NY", "Remote", "Austin, TX"], "salary": [95000, 160000, None], }) df.shape # (3, 3) df.dtypes # object, object, float64 - one dtype per column df.index # RangeIndex(start=0, stop=3) - the shared index

The index is not a column The index labels the rows; it sits to the left of your data. Beginners often try to select it like a column and get confused. Reach the index with df.index, and reset it to a plain column with df.reset_index() when you need it as data.

Part 2 · covers pandas Indexing and selecting data

Selecting: loc, iloc, and columns 5 min live

This is the one thing every pandas beginner trips on, so we slow down. There are two ways to pull rows out: .loc works by label, .iloc works by integer position. Confuse them and you either get an error or, worse, the wrong rows silently.

🔍 Click to zoom - loc selects by label, iloc selects by integer position

LiveColumns first, then loc vs iloc3 min▶

Selecting a column is the everyday move: df["salary"] returns a Series, and a list of names df[["title", "salary"]] returns a smaller DataFrame. Note the double brackets - a list inside the brackets - whenever you want more than one column back as a table.

★ Columns, then rows two waysdf["salary"] # one column -> a Series df[["title", "salary"]] # a list of columns -> a DataFrame df.loc[0, "salary"] # label-based: row labeled 0, salary column df.iloc[0, 3] # position-based: 1st row, 4th column df.loc[0:4, ["title", "salary"]] # loc slice is INCLUSIVE of 4 df.iloc[0:5, 0:2] # iloc slice EXCLUDES 5 (like Python)

The gotcha that bites everyone .loc slices are inclusive of the end label; .iloc slices exclude the end position, just like normal Python. df.loc[0:4] gives five rows; df.iloc[0:4] gives four. When in doubt, print the shape.

Self-studyThe selection pattern table2 min read▶

You want	Write	Returns
One column	df["salary"]	Series
Several columns	df[["title", "salary"]]	DataFrame
One cell by label	df.loc[10, "salary"]	a scalar
Rows + cols by label	df.loc[0:4, ["title"]]	DataFrame (end inclusive)
Rows + cols by position	df.iloc[0:5, 0:2]	DataFrame (end excluded)
Rows matching a condition	df.loc[df["remote"]]	filtered DataFrame

Part 3 · covers Kaggle Pandas L2-L3 + McKinney ch5

Filtering, summaries, and maps 4 min live

Now the payoff. Filtering is the Session 2 boolean mask, now on a DataFrame. Summary functions collapse a column to an answer. And map/apply transform a column into a new one. Put them together and you can interrogate the LinkedIn data.

LiveBoolean filtering - and the parentheses rule2 min▶

Pass a boolean mask into the brackets and pandas keeps only the rows where it's True. Combine conditions with & (and) and | (or) - never Python's and/or - and wrap each condition in parentheses, because & binds tighter than > and will misfire without them.

★ Filter with combined conditions# each condition in its own parentheses, joined by & / | high_remote = df[(df["salary"] > 100000) & (df["remote"])] # "or" example di = df[(df["title"].str.contains("Data")) | (df["title"].str.contains("ML"))] # membership - cleaner than chaining many == big3 = df[df["location"].isin(["New York, NY", "Remote", "Austin, TX"])]

Don't - Python and/or on Series raises an errordf[df["salary"] > 100000 and df["remote"]] # ValueError: ambiguous truth value # use & with parentheses, not and

LiveSummary functions + map/apply2 min▶

Summary functions turn a column into an answer: .mean(), .describe() (a whole stats block at once), .unique() and .nunique() (distinct values and how many), and the analyst's favorite, .value_counts(), which tallies each distinct value sorted from most to least common.

★ Summaries + a derived columndf["title"].value_counts().head(10) # top 10 job titles by frequency df["salary"].describe() # count, mean, std, min, quartiles, max df["location"].nunique() # how many distinct locations? # map/apply build a new column from an old one df["remote"] = df["location"].map(lambda x: x == "Remote") df["pay_band"] = df["salary"].apply(lambda s: "high" if s > 120000 else "standard")

Real world

Run value_counts on the LinkedIn title column and you meet the dataset's messiest reality: "Data Analyst", "Data analyst", "Sr. Data Analyst", and "Data Analyst II" all count separately. Seeing that tally is the moment you understand why Session 4 spends real time normalizing titles - the counts lie until you clean them.

Demo 1 of 3

Load and build a working subset ★ 8 min · everyone builds

Load postings.csv, then cut it down to the columns we actually care about this session.

Inspect the subset: dtypes and head. Then practice iloc vs loc on it live.

★ Load + subset the columnsimport pandas as pd jobs = pd.read_csv("postings.csv") cols = ["title", "company_name", "location", "max_salary", "min_salary", "pay_period"] work = jobs[cols].copy() # .copy() so we don't touch the raw frame work.dtypes # what type is each column? salary numeric, title object work.head() # eyeball the first five rows

★ iloc vs loc, side by sidework.iloc[0] # the FIRST row, by position -> a Series work.loc[0] # the row LABELED 0 - same here, since index is 0..N work.iloc[0:5, 0:3] # first 5 rows, first 3 cols (end excluded) work.loc[0:4, ["title", "location"]] # rows 0..4 inclusive, named cols

Always .copy() a subset you'll modify Slicing without .copy() can trigger pandas' "SettingWithCopyWarning" when you later assign into it. Copying up front keeps the raw frame pristine (a Session 1 habit) and silences the warning.

Demo 2 of 3

Filter to data roles, count the leaders ★ 8 min · everyone builds

This is the first real "answer a question" moment. Filter the postings down to data / analytics / AI roles by matching the title text.

Then value_counts the top titles and top locations in that slice.

★ Filter by title text, then count# case-insensitive match on the messy free-text title mask = work["title"].str.contains("data|analyst|machine learning|ml ", case=False, na=False) data_jobs = work[mask] data_jobs.shape # how many data-ish postings? data_jobs["title"].value_counts().head(10) # top 10 titles data_jobs["location"].value_counts().head(10) # top 10 hiring locations

Real world

The na=False matters: some titles are missing, and without it str.contains returns NaN for those rows and the mask breaks. This is the everyday texture of real data work - a tiny flag that separates code that runs from code that errors on row 5,000. You'll hit this exact pattern all course long.

Demo 3 of 3

Build a derived column ★ 5 min · follow along

Create a new column from existing ones and assign it back with df["newcol"] = .... We'll flag remote roles.

★ Derive and assign a new column# flag remote roles from the location text work["is_remote"] = work["location"].str.contains("remote", case=False, na=False) work["is_remote"].value_counts() # how many remote vs on-site? # bucket a numeric column with apply work["pay_band"] = work["max_salary"].apply( lambda s: "high" if s > 120000 else "standard" ) work[["title", "is_remote", "pay_band"]].head()

Vectorized beats apply when you can str.contains(...) is vectorized and fast (it's NumPy underneath - Session 2). .apply(lambda ...) loops row by row in Python, so save it for logic that has no vectorized form. On 124k rows the difference is noticeable.

After the session

This week ◐ 40 min total

Do Kaggle Pandas lessons 1-3 exercises (Creating/Reading/Writing, Indexing & Selecting, Summary Functions & Maps). They're free and mirror today exactly.
Answer the headline question on the real data: what are the top 10 job titles and the top 10 hiring locations across all postings? Save the two value_counts outputs - Session 6 will chart them.
Practice loc vs iloc until it's automatic: pull the same three cells three different ways and confirm they match.
Filter twice: once with a numeric threshold on salary, once combining two conditions with & and parentheses. Print the shape each time.
Read McKinney ch5 (Getting Started with pandas) - the canonical, free, open-access reference for everything today.

Check yourself

Three questions before you go 🎯 ◐ 90 seconds

1 · What's the difference between .loc and .iloc?

loc reads like a name (row labeled 0), iloc reads like a counter (the first row). Boolean masks go with loc.

2 · How do you keep rows where salary > 100000 AND the role is remote?

On Series you use & / |, not Python's and/or, and each condition needs its own parentheses because & binds tightly.

3 · What does value_counts() do?

It's the fastest way to see the shape of a categorical column - and on the messy title column it also reveals why cleaning is coming in Session 4.

Source material

Official sources covered

This session teaches the working content of the free Kaggle Pandas micro-course, the official pandas user guide, and Wes McKinney's open-access Python for Data Analysis (3e). Certificates and graded exercises stay on those platforms - links provided.

✓Kaggle Pandas L1-L3creating/reading, indexing & selecting, summary functions & maps - all three parts

✓pandas "10 minutes to pandas" + Indexing and selecting dataSeries/DataFrame, loc/iloc, boolean filtering - Parts 1-3

✓McKinney ch5 - Getting Started with pandasthe canonical reference for today's structures and selection

◐MultiIndex / hierarchical indexingintro only - covered properly with groupby in Session 5

Session 3 cheat sheet · pin this

Series vs DataFrameSeries = one labeled column (index + values, one dtype). DataFrame = dict of Series sharing an index.

Column selectdf["col"] -> Series · df[["a","b"]] -> DataFrame (double brackets for a list of columns).

loc vs ilocdf.loc[label, "col"] by label (end inclusive) · df.iloc[pos, n] by integer position (end excluded).

Boolean filterdf[(df.a > x) & (df.b)] - use & | with parentheses around each condition, plus .isin([...]).

Summary functionsvalue_counts() tallies categories · describe() a stats block · nunique()/unique() distinct values.

map / apply + assigndf["new"] = df["old"].map(fn) or .apply(fn) - build a derived column, then assign it back.