Up to this point, we have focused on understanding data.
We inspected a table, cleaned it, transformed it, visualized it, summarized it, and wrote careful insights.
This is where many learning paths stop.
In CDI, this point is also a handoff.
The Data Science Foundations guide prepares analysis-ready data and supporting evidence. The advanced data science guide, maintained in a separate repository, builds on those outputs when machine learning, evaluation, interpretation, or decision support is needed.
This chapter introduces that transition. It does not attempt to teach the advanced workflow.
Data analysis helps us answer questions such as:
Modeling extends the workflow further.
It helps us ask questions such as:
Modeling depends on analysis.
Clean, well-structured, well-understood data is what makes modeling possible.
The foundations workflow produces a compact set of reusable outputs:
data/iris.csv
data/iris_clean.csv
data/iris_wrangled.csv
results/inspection/
results/cleaning/
results/wrangling/
results/figures/
results/summary/These outputs document the journey from raw teaching table to analysis-ready evidence package.
The handoff point looks like this:
Data Science Foundations
↓
tidy, cleaned, wrangled table
↓
figures and summary evidence
↓
Advanced Data Science
↓
modeling, evaluation, interpretation, decision-makingThe important idea is that modeling should not begin from an unexplored table.
It should begin from a table whose structure, quality, and interpretation have already been examined.
flowchart TD
A[Raw or Source Data] --> B[Inspect and Clean]
B --> C[Wrangle and Validate]
C --> D[Visualize and Summarize]
D --> E[Interpret Findings]
E --> F[Assess Modeling Readiness]
F --> G[Advanced Data Science]flowchart TD
A[Raw or Source Data] --> B[Inspect and Clean]
B --> C[Wrangle and Validate]
C --> D[Visualize and Summarize]
D --> E[Interpret Findings]
E --> F[Assess Modeling Readiness]
F --> G[Advanced Data Science]
This workflow shows why the foundations are reusable across CDI pathways.
Any pathway that produces a tidy table can enter this workflow.
Examples:
Proteomics result table
Microbiome diversity table
Clinical cohort table
AI evaluation tableEach can move through the same foundational process before advanced work begins.
Modeling introduces new responsibilities.
A model is not just a calculation. It is a system component that may influence interpretation, prioritization, or decisions.
Modeling adds questions about:
These topics belong in the separate advanced data science guide.
This foundations chapter introduces the handoff without teaching the full modeling workflow.
Before moving into modeling, ask whether the data is ready.
Modeling readiness checklist
[ ] Is the input table tidy?
[ ] Are required columns present?
[ ] Are missing values understood or handled?
[ ] Are duplicates checked?
[ ] Are feature types clear?
[ ] Is the target variable clearly defined?
[ ] Have relevant patterns and group differences been explored?
[ ] Are summaries and figures available to support interpretation?
[ ] Are limitations documented?
[ ] Is there enough data for a defensible train/test strategy?
[ ] Could any feature reveal the target or information from the future?If several items are missing, return to inspection, cleaning, wrangling, or summarization before modeling.
The following display-only example previews what the next stage may look like. It is intentionally brief and is not executed as part of this guide.
The goal is not to teach machine learning here. It is simply to show how the wrangled Iris table could become model input.
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import classification_report
df = pd.read_csv("data/iris_wrangled.csv")
features = [
"sepal_length",
"sepal_width",
"petal_length",
"petal_width",
"petal_area"
]
X = df[features]
y = df["species"]
X_train, X_test, y_train, y_test = train_test_split(
X,
y,
test_size=0.25,
random_state=42,
stratify=y
)
model = RandomForestClassifier(
n_estimators=200,
random_state=42
)
model.fit(X_train, y_train)
predictions = model.predict(X_test)
print(classification_report(y_test, predictions))This preview introduces several new concepts:
Each of these requires careful treatment in the advanced data science guide.
One train/test split is only a preview. It is not enough to establish that a model is reliable. Model choice, cross-validation, baselines, leakage checks, uncertainty, interpretation, and responsible use belong to the advanced workflow.
A model becomes more useful when it is part of a larger system.
flowchart TD
A[Analysis-Ready Table] --> B[Model]
B --> C[Evaluation]
C --> D[Interpretation]
D --> E[Decision Support]flowchart TD
A[Analysis-Ready Table] --> B[Model]
B --> C[Evaluation]
C --> D[Interpretation]
D --> E[Decision Support]
In more advanced settings, the workflow may extend further:
flowchart TD
A[Build] --> B[Test]
B --> C[Deploy]
C --> D[Monitor]
D --> E[Improve]
E --> Aflowchart TD
A[Build] --> B[Test]
B --> C[Deploy]
C --> D[Monitor]
D --> E[Improve]
E --> A
This is why the next CDI layer is not just about algorithms.
It is about applied analytical systems.
The separate advanced guide can build on these foundations with topics such as:
This creates a clear separation of scope:
Data Science Foundations
= inspect, clean, wrangle, visualize, summarize, interpret tidy tables
Advanced Data Science
= model, evaluate, explain, improve, and support decisionsBoth belong to the broader Data Science pathway, but they are maintained as separate guides and repositories.
The Foundations System is reusable because many CDI pathways eventually produce tidy tables.
Omics Pathway
↓
differential results, abundance tables, diversity metrics
Clinical & Medical Data Pathway
↓
cohort tables, lab tables, outcome tables
Human-Centered AI
↓
evaluation tables, prediction logs, decision recordsOnce these tables are tidy and analysis-ready, they can use the same foundations:
inspect
clean
wrangle
visualize
summarize
interpretWhen modeling is needed, the workflow continues in the advanced data science guide.
Review the outputs produced so far:
data/iris_wrangled.csv
results/figures/
results/summary/analysis-insights.mdThen answer:
A reasonable classification target is:
speciesPotential input features include:
sepal_length
sepal_width
petal_length
petal_width
petal_areaThe strongest exploratory support comes from petal-related plots and grouped summaries, which show clearer descriptive separation among species. Because petal_area is calculated from petal_length and petal_width, it is also important to assess redundancy and decide whether the derived feature adds useful information.
A careful caution is:
The Iris dataset is small, clean, and educational. Performance from one split should not be treated as evidence that the model will generalize to larger, noisier, real-world datasets. Repeated evaluation, suitable baselines, and careful validation are still required.Data analysis is not the final step.
It is the foundation for building reliable analytical systems.
A model built without inspection, cleaning, visualization, and summary interpretation is fragile.
A model built after careful data science foundations is easier to understand and evaluate. Trust still depends on appropriate validation, transparent limitations, and responsible use.
In this lesson, you:
You have now completed the Data Science Foundations workflow:
load → inspect → clean → wrangle → visualize → summarize → interpretYou can return to the appendix whenever you need supporting commands, terminology, or reference material. The appendix is a reference resource rather than another lesson.
When you are ready to continue beyond foundations, move to the separate advanced data science guide. That guide begins where this one ends and develops machine learning, model evaluation, interpretation, and decision support in greater depth.
The most important outcome of this guide is not a single dataset or technique. It is a reusable way of working with data carefully, clearly, and reproducibly.