Scikit-Learn Guide: Build Your First ML Models with Python

Scikit-learn provides a consistent way to turn tabular Python data into a trained and evaluated machine learning model, fitting into a broader data analysis and model building workflow. This guide follows one small classification dataset from beginning to end, keeping the focus on both the code and the decisions that make the result reliable as a beginner-friendly machine learning roadmap.

The example uses scikit-learn’s built-in breast cancer dataset. It is suitable for learning the workflow, not for making medical decisions.

Set Up a Small scikit-learn Project and Choose a Learning Task

Begin with a clean Python environment. You can use a notebook such as Jupyter or put the code in a file such as first_model.py.

Use Python 3.10 or later. The examples target scikit-learn 1.7.2, which is pinned here so the environment is explicit.

Install the packages if they are not already available.

python -m pip install "scikit-learn==1.7.2" pandas matplotlib

Load a dataset and inspect its contents.

import pandas as pd
from sklearn.datasets import load_breast_cancer

dataset = load_breast_cancer(as_frame=True)

X = dataset.data
y = dataset.target

print("Feature table shape:", X.shape)
print("\nFeature names:")
print(X.columns.tolist())

print("\nClass labels:")
print(dict(enumerate(dataset.target_names)))

print("\nFirst five rows:")
print(X.head())

print("\nClass counts:")
print(y.value_counts().sort_index())

In this example:

X contains the features: the input columns available to the model.
y contains the target: the answer the model is expected to predict.
Each row represents one sample described by numerical measurements.
The target has two classes: 0 for malignant and 1 for benign.

Keeping X and y separate is important throughout the project. The features go into the model, while the target stays apart so you can later compare predictions with the known labels.

This dataset is convenient because it already comes as a numeric table. Real-world data is rarely so tidy. You may encounter blank cells, text categories, inconsistent units, or a column with a name as unhelpful as final_final_version2.

Real story

I once spent an hour tuning a scikit-learn model and felt very serious about it, right up until I printed the predictions and saw a row of identical answers. My laptop was open, my notebook was full of notes, and the model had basically chosen one favorite class and committed to the bit. I leaned back, stared at the screen, and said, "So we're doing vibes-based classification today."

Have a story of your own? Share it in the comments below.

Prepare Features and Split Data Without Leaking the Answers

Before fitting a model, reserve some rows that it will not see during training or model selection. Those rows become the final test set.

Check the data structure.

print("Missing values:", X.isna().sum().sum())

print("\nColumn data types:")
print(X.dtypes.value_counts())

categorical_columns = X.select_dtypes(
    include=["object", "category"]
).columns.tolist()

print("\nCategorical columns:", categorical_columns)

For this dataset, all features are numeric, and there should be no missing values. A typical business dataset might instead reveal text categories such as "region" or missing values in a column such as "income".

Split the data into training and test sets.

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(
    X,
    y,
    test_size=0.20,
    random_state=42,
    stratify=y
)

print("Training features:", X_train.shape)
print("Test features:", X_test.shape)
print("Training labels:", y_train.shape)
print("Test labels:", y_test.shape)

print("\nTraining class proportions:")
print(y_train.value_counts(normalize=True).sort_index())

print("\nTest class proportions:")
print(y_test.value_counts(normalize=True).sort_index())

random_state=42 makes the split repeatable. With the same dataset, code, and compatible Python and package versions in the same environment, you should get the same split.

stratify=y keeps the class balance similar in both groups. That is particularly useful for classification tasks in which one class is less common than the other.

Keep the Test Set Out of Preparation and Selection

Leave the test set alone until you have chosen a final model. Do not fit a scaler, fill missing values, select features, compare models, tune settings, or inspect test-set errors using the training and test data together.

Suppose one feature ranges from 0 to 1 while another reaches into the thousands. A scaler must calculate values such as the mean and standard deviation. Those calculations should use X_train, not the complete dataset.

Scikit-learn pipelines help enforce this separation. The next section uses one.

Build and Evaluate a Baseline on Training Folds

A baseline is a reasonable first model, not a final verdict. It gives you a reference point before you change settings or compare other approaches.

Logistic regression is a useful baseline for many binary classification problems. Despite its name, it is a classification model.

Create a pipeline that scales the features and trains a logistic regression model.

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression

baseline_model = Pipeline(
    steps=[
        ("scaler", StandardScaler()),
        ("model", LogisticRegression(max_iter=1000))
    ]
)

The pipeline contains two steps:

StandardScaler puts numeric features on comparable scales.
LogisticRegression learns a classification rule from the scaled training data.

Define cross-validation using only the training data.

from sklearn.model_selection import StratifiedKFold, cross_validate

cv = StratifiedKFold(
    n_splits=5,
    shuffle=True,
    random_state=42
)

Five-fold stratified cross-validation divides X_train and y_train into five folds. On each run, the model fits on four folds and validates on the remaining fold. X_test and y_test stay out of the process.

Evaluate the baseline with cross-validation.

baseline_results = cross_validate(
    baseline_model,
    X_train,
    y_train,
    cv=cv,
    scoring={
        "accuracy": "accuracy",
        "macro_f1": "f1_macro"
    }
)

print(
    "Baseline mean cross-validation accuracy:",
    round(baseline_results["test_accuracy"].mean(), 3)
)
print(
    "Baseline mean cross-validation macro F1:",
    round(baseline_results["test_macro_f1"].mean(), 3)
)

Accuracy is the proportion of correct predictions. Macro F1 calculates an F1 score for each class and averages the scores equally, which can be useful when both classes matter.

The central scikit-learn pattern looks like this:

model = SomeEstimator(...)
model.fit(X_train, y_train)
predictions = model.predict(X_unseen)

During development, X_unseen can be a validation fold taken from the training set. In this workflow, keep X_test for a single final evaluation after model selection.

Responding to Convergence Warnings

Logistic regression may warn that it did not converge within the allowed number of iterations. Do not simply suppress the warning and continue.

Check these points first:

Confirm that numeric features are scaled.
Increase max_iter moderately.
Check for missing values, duplicate columns, or unusually large values.
Consider whether the selected solver fits the dataset.

For this tutorial, max_iter=1000 is a reasonable starting point. More iterations are not inherently better; they only give the optimization process more time to finish.

Improve the Baseline Through Controlled Model Iteration

Model improvement works best as a small, documented experiment rather than a contest between dozens of guesses. Keep the training split fixed, choose the metric that matters, compare a limited set of options, and use cross-validation to guide the decision.

The example below uses the five-fold stratified cross-validation strategy defined earlier and selects models by macro F1 score.

Macro F1 calculates an F1 score for each class and gives the classes equal weight. It can be more informative than accuracy when performance on both classes matters.

Compare a scaled logistic regression model with a random forest.

from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import GridSearchCV

search_pipeline = Pipeline(
    steps=[
        ("scaler", StandardScaler()),
        ("model", LogisticRegression(max_iter=2000))
    ]
)

parameter_grid = [
    {
        "model": [LogisticRegression(max_iter=2000)],
        "model__C": [0.1, 1.0, 10.0]
    },
    {
        "scaler": ["passthrough"],
        "model": [RandomForestClassifier(random_state=42, n_jobs=1)],
        "model__n_estimators": [200, 400],
        "model__max_depth": [None, 10]
    }
]

search = GridSearchCV(
    estimator=search_pipeline,
    param_grid=parameter_grid,
    scoring="f1_macro",
    cv=cv,
    n_jobs=-1,
    refit=True
)

search.fit(X_train, y_train)

print("Best cross-validation macro F1:", round(search.best_score_, 3))
print("Best parameters:")
print(search.best_params_)

GridSearchCV tests each listed configuration across the cross-validation folds. Each fold uses one portion of X_train for fitting and another for validation. The held-out X_test data is not involved.

The pipeline handles the two model families differently:

Logistic regression receives scaled features.
Random forest uses the original numeric features because tree-based models generally do not need scaling.

This is intentionally a small search. A very large parameter grid can take considerable time and may tempt you to optimize around tiny score differences.

Evaluate the selected model on the untouched test set.

import matplotlib.pyplot as plt
from sklearn.metrics import (
    accuracy_score,
    classification_report,
    ConfusionMatrixDisplay
)

final_model = search.best_estimator_
final_predictions = final_model.predict(X_test)

final_accuracy = accuracy_score(y_test, final_predictions)

print(f"Final test accuracy: {final_accuracy:.3f}")
print()

print(
    classification_report(
        y_test,
        final_predictions,
        target_names=dataset.target_names,
        digits=3
    )
)

The classification report provides metrics for each class:

Accuracy is the proportion of all predictions that were correct.
Precision asks: when the model predicts a class, how often is that prediction correct?
Recall asks: among the rows that truly belong to a class, how many did the model identify?
F1 score combines precision and recall into one measure.

Accuracy can conceal important errors when the classes are heavily imbalanced. If 98% of transactions are legitimate, for instance, a model that always predicts “legitimate” would achieve 98% accuracy while finding no fraud. That is technically accurate but practically useless.

Display the final confusion matrix.

ConfusionMatrixDisplay.from_predictions(
    y_test,
    final_predictions,
    display_labels=dataset.target_names,
    cmap="Blues"
)

plt.title("Selected Model Confusion Matrix")
plt.show()

A confusion matrix shows the counts behind the summary metrics:

Rows are the actual labels.
Columns are the model predictions.
Values on the diagonal are correct predictions.
Values outside the diagonal are mistakes.

For this dataset, class 0 is malignant and class 1 is benign. When malignant is treated as the condition of interest, a malignant row predicted as benign is a false negative. A benign row predicted as malignant is a false positive.

The more serious error depends on the real task. In a medical setting, the consequences require expert review, careful validation, and much more than a tutorial dataset can provide. Here, the purpose is simply to learn how to interpret the output.

At this stage, the test score estimates how the selected model may perform on similar unseen data. It does not guarantee performance on every future dataset.

Save the details of the experiment.

Record the split, metric, cross-validation setup, selected parameters, library version, and final result. That record makes future comparisons much easier.

import json
from pathlib import Path
import sklearn

experiment_details = {
    "random_state": 42,
    "test_size": 0.20,
    "cv_folds": 5,
    "selection_metric": "f1_macro",
    "scikit_learn_version": sklearn.__version__,
    "best_cv_score": float(search.best_score_),
    "best_parameters": {
        key: str(value)
        for key, value in search.best_params_.items()
    },
    "final_test_accuracy": float(final_accuracy)
}

Path("experiment_details.json").write_text(
    json.dumps(experiment_details, indent=2)
)

For later model-development experiments, examine the cross-validation results and training-set validation errors before changing the approach. Look for rows that are repeatedly misclassified, patterns in missing data, poorly measured features, or a metric that does not reflect the cost of mistakes. Do not use the final test results to choose another model. If more selection is necessary, use a new untouched test set.

A dependable first scikit-learn workflow is straightforward: inspect the data, split it carefully, put preprocessing in a pipeline, evaluate a baseline with training-set cross-validation, compare options under the same cross-validation setup, and test the selected model once. That approach is more useful than swapping models at random until one produces an appealing number.

Scikit-Learn: A Practical Guide to Building Your First Machine Learning Models