Pandas GroupBy and Merge: The Two Skills That Cover Most Analysis Work

If you sat down and logged what you do in pandas over a month, two operations would dominate almost everything else: splitting data into groups and summarising it, and sticking two tables together. Everything else is decoration. Get these two right and you can do most analysis work. The problem is that both of them … Read more

Data Cleaning With Pandas: A Step by Step Walkthrough on a Messy File

Every pandas tutorial starts with a clean dataset. Real work never does. Real work starts with a file called sales_final_v3_USE_THIS.xlsx where the header is on row four, prices are stored as text with currency symbols, half the dates are in the wrong format, and the region column contains “North”, “north”, “N” and an empty cell. … Read more

Cleaning a Client’s Messy Data With Python: A Complete Project Walkthrough

data cleaning

The brief The client came to me with a simple sounding problem: “our sales numbers do not match and we do not know why.” They had been running the business for a few years. Data had accumulated in the way it usually does, which is to say one file per month, exported from different systems … Read more

Automating the Report Nobody Wants To Run: A Practical Guide

There is a specific kind of Monday morning misery familiar to anyone who has worked in analytics. Open the laptop, run four queries, paste the results into a spreadsheet, fix the formatting because the date column came through as text again, write a short email, attach the file, send. Forty minutes. Every week. Forever. The … Read more