Appendix

Published

Aug 2026

  • ID: DS-APP
  • Type: Reference
  • Audience: Public
  • Theme: Data science foundations quick reference

This appendix brings together the commands, project structure, and reusable Python patterns introduced throughout the guide. Use it as a quick reference when starting a new analysis.

Project Structure

A small reproducible data science project may use the following structure:

data-science/
├── data/
│   ├── raw/
│   └── processed/
├── results/
│   ├── figures/
│   └── tables/
├── scripts/
│   ├── bash/
│   └── python/
├── .gitignore
├── _quarto.yml
├── index.qmd
└── requirements.txt
  • data/raw/ stores unchanged source data.
  • data/processed/ stores cleaned or transformed data.
  • results/figures/ stores saved visualizations.
  • results/tables/ stores summary tables and other tabular outputs.
  • scripts/python/ stores reusable analysis scripts.
  • scripts/bash/ stores setup and build helpers.
  • requirements.txt records the Python packages required by the project.

Keeping inputs, code, and outputs separate makes an analysis easier to understand, reproduce, and update.

Environment Commands

Create and activate a virtual environment

python3 -m venv .venv
source .venv/bin/activate

On Windows PowerShell, activate the environment with:

.venv\Scripts\Activate.ps1

Install project packages

python -m pip install -r requirements.txt

Run a Python script

python scripts/python/script_name.py

Render the Quarto guide or report

quarto render

Reusable pandas Patterns

The examples below assume that pandas has been imported and a DataFrame named df already exists.

import pandas as pd

Inspect a dataset

df.head()
df.shape
df.info()
df.describe(include="all")

Select columns

df[["col1", "col2"]]

Filter rows

df.loc[df["col"] > value]

Rename columns

df = df.rename(columns={"old_name": "new_name"})

Convert a data type

df["numeric_col"] = pd.to_numeric(df["numeric_col"], errors="coerce")
df["category_col"] = df["category_col"].astype("category")

Identify missing values

df.isna().sum()

Fill missing numeric values

df["numeric_col"] = df["numeric_col"].fillna(
    df["numeric_col"].median()
)

Identify duplicate rows

df.duplicated().sum()

Group and summarize

summary = (
    df.groupby("group_col", observed=True)
    .agg(
        count=("value_col", "size"),
        mean=("value_col", "mean"),
        median=("value_col", "median"),
    )
    .reset_index()
)

Save a processed dataset

df.to_csv("data/processed/cleaned_data.csv", index=False)

Data Validation Checks

Before interpreting results, confirm that:

  • rows and columns have the expected meaning;
  • column names are clear and consistent;
  • numeric, categorical, and text columns use appropriate data types;
  • missing values have been identified and handled deliberately;
  • duplicate rows have been investigated;
  • categorical values use consistent spelling and labels;
  • numeric values fall within plausible ranges;
  • cleaned data and analytical outputs are saved separately from raw data.

These checks do not guarantee that a dataset is correct, but they make common data-quality problems visible before they affect the analysis.

Reproducibility Checklist

Before sharing an analysis:

  • preserve the original data unchanged;
  • record package dependencies in requirements.txt;
  • use relative paths within the project;
  • keep reusable code in scripts;
  • save cleaned data, tables, and figures to clearly named folders;
  • rerun the workflow from a clean environment when practical;
  • review tables and figures for correctness;
  • document important cleaning and analytical decisions.

Data Source

Software and Tools

Together, these tools provide a practical foundation for reproducible data analysis in Python.