Data analysis (Pandas) interview questions

loc selects by index labels and column names, and its slices include the end label. iloc selects by integer position and excludes the end, like Python lists. I use loc for filters such as df.loc[df["status"] == "paid", "amount"] and iloc when I need the first or last rows regardless of the index.

First I measure them with df.isna().sum() and look at whether they are random or tied to a source, such as one branch that never fills a field. Then I decide per column: drop rows if few are affected, fill with a sensible value such as 0 for no sales or the median for a numeric field, or keep them and flag them. I write down the choice so the result can be reproduced.

The key is almost certainly duplicated in the right table, so each left row matched more than once. I check with right["key"].duplicated().sum(), then either remove duplicates or aggregate the right table first. On important joins I pass validate="many_to_one" so pandas raises an error instead of silently inflating totals.

I avoid apply when a vectorised operation exists, such as column arithmetic, str methods or np.where, because those run much faster on large data. I use apply for logic that does not fit a built-in method, for example parsing an irregular text field with a custom function. If performance matters, I test both on a sample.

Both aggregate by groups. groupby returns a long result indexed by the group keys, which is good for further processing. pivot_table spreads one key across columns to make a grid, which suits reports, for example regions as rows and months as columns with summed sales.

I compare row counts before and after each join or filter, and check that totals still add up to the source total. I also recalculate one or two groups by hand or in a spreadsheet. For example, if the report says region North sold 4,200 units, I filter the raw data for North and sum it to confirm.