Feature Engineering for Machine Learning: Techniques, Examples, and a Reusable Workflow
feature-engineeringdata-preprocessingscikit-learnmachine-learningdata-science

Feature Engineering for Machine Learning: Techniques, Examples, and a Reusable Workflow

SSkilling.pro Editorial Team
2026-11-20
7 min read

A practical feature engineering checklist for missing values, categories, dates, text, leakage prevention, and reusable scikit-learn pipelines.

Feature engineering turns raw columns into signals a machine learning model can use. This practical checklist explains how to handle missing values, categories, dates, text, scaling, high-cardinality data, leakage, and train-test consistency with a reusable Python workflow.

Overview

Feature engineering is the process of creating, transforming, or selecting input variables for a machine learning model. A feature might be a cleaned version of an existing column, such as a numeric value with missing entries filled, or a new signal derived from several fields, such as the number of days since a customer’s last order.

Good feature engineering does not mean adding as many columns as possible. The aim is to represent information in a form that is available at prediction time, meaningful for the task, and compatible with the model and deployment workflow. A small number of well-designed features can be more useful than a large collection of noisy transformations.

A dependable workflow usually follows this order:

  1. Define the prediction target and the moment when a prediction will be made.
  2. Inspect data types, missingness, duplicate records, unusual values, and cardinality.
  3. Split data using a method appropriate to the problem.
  4. Fit transformations only on the training data.
  5. Apply the same transformations to validation, test, and production data.
  6. Compare feature changes using the same evaluation procedure.
  7. Document the final feature definitions and their data requirements.

The prediction timestamp is especially important. It provides a boundary between information that would have been available and information that would not. This boundary helps prevent target leakage, one of the most damaging errors in machine learning data preprocessing.

Technique comparison

Data situationUseful techniquesPrimary caution
Missing numeric valuesMedian imputation, missingness indicators, domain-based replacementDo not calculate imputation values from validation or test data
Low-cardinality categoriesOne-hot encoding, ordinal encoding when order is realHandle unseen categories consistently
High-cardinality categoriesFrequency encoding, hashing, carefully validated target encodingTarget encoding can leak the label
Dates and timestampsYear, month, weekday, elapsed time, cyclical componentsRespect time order and avoid future information
TextTF-IDF, token counts, embeddings, domain-specific featuresFit the vocabulary only on training data
Different numeric scalesStandardization, robust scaling, min-max scalingScaling is essential for some models but unnecessary for others

Checklist by scenario

1. Missing numeric values

Start by measuring missingness per column and checking whether missing values have a plausible meaning. Median imputation is a practical baseline for skewed numeric data because it is less affected by extreme values than the mean. Add a missingness indicator when the fact that a value is absent may itself carry information.

  • Check whether missingness is concentrated in a particular group or time period.
  • Fit the imputer on training data only.
  • Use a consistent fallback for new rows in production.
  • Confirm that placeholder values such as -1 or 999 are not being treated as genuine measurements.

2. Categorical variables

For small, stable sets of categories, one-hot encoding is a clear starting point. Use ordinal encoding only when the categories have a genuine order, such as a defined education or service tier. Arbitrary numeric codes can falsely imply that one category is larger or closer to another.

For categories that may appear after training, configure the preprocessing step to handle unknown values rather than failing at inference. For high-cardinality columns such as product IDs, account IDs, or postal codes, first ask whether the field represents a reusable pattern or merely identifies a row. An identifier often adds noise or memorization risk.

3. Dates and time-based data

Dates can provide several useful features: calendar year, month, day of week, hour, elapsed time since an event, or time since the previous transaction. Choose features based on the process being modeled. A delivery problem may benefit from weekday and elapsed processing time; a demand problem may need seasonal and holiday-related signals when those inputs are available at prediction time.

For periodic variables such as hour or month, sine and cosine transformations can represent the fact that the end and beginning of a cycle are close. For forecasting and other temporal tasks, use chronological validation rather than a random split. Never use a future aggregate, later status, or post-outcome field to predict an earlier event.

4. Text features

For a first text classification model, TF-IDF features are interpretable and straightforward to test. Clean text consistently, define tokenization rules, and decide how to handle punctuation, numbers, casing, and very rare terms. Fit the vocabulary on training data only. If you are working through a complete text workflow, pair feature engineering with an error-analysis process; the text classification tutorial provides a useful adjacent workflow.

5. Numeric scale and distributions

Scaling is particularly important for distance-based models, linear models with regularization, neural networks, and algorithms that use gradients. Tree-based models generally do not require standardization, although consistent preprocessing can still simplify a shared pipeline. For strongly skewed variables, consider a log-like transformation where values are non-negative and the transformation makes domain sense. Inspect the result rather than applying transformations automatically.

6. Reusable scikit-learn pipeline

Keep preprocessing attached to the estimator so that training and inference use the same operations. A compact tabular pattern looks like this:

from sklearn.compose import ColumnTransformer
from sklearn.impute import SimpleImputer
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler
from sklearn.ensemble import RandomForestClassifier

numeric_pipe = Pipeline([
    ("imputer", SimpleImputer(strategy="median", add_indicator=True)),
    ("scaler", StandardScaler())
])

categorical_pipe = Pipeline([
    ("imputer", SimpleImputer(strategy="most_frequent")),
    ("encoder", OneHotEncoder(handle_unknown="ignore"))
])

preprocessor = ColumnTransformer([
    ("numeric", numeric_pipe, numeric_columns),
    ("categorical", categorical_pipe, categorical_columns)
])

model = Pipeline([
    ("preprocessor", preprocessor),
    ("classifier", RandomForestClassifier(random_state=42))
])

model.fit(X_train, y_train)

The exact estimator and column lists will vary, but the principle is stable: fit feature transformations inside the training workflow, not as a separate manual step that can be applied differently later.

What to double-check

  • Leakage: Could this feature include the label, a later event, or a statistic calculated using the entire dataset?
  • Availability: Will the same field exist, with the same delay and quality, when the model is used?
  • Split design: Does the validation strategy match the real task, particularly for users, groups, duplicates, or time?
  • Train-test consistency: Are column names, category handling, units, and missing-value rules identical across datasets?
  • Feature meaning: Can you explain what each important feature represents and why it might help?
  • Evaluation: Did the new feature improve the metric on an untouched test set, or only on a repeatedly inspected validation set?
  • Operational cost: Can the feature be computed within the prediction latency, storage, and maintenance limits of the application?

Keep a feature log with the feature name, source columns, transformation, prediction-time availability, owner, and validation notes. When comparing iterations, change one logical group of features at a time where practical. Record both performance and failure patterns; a feature that improves an aggregate metric may still worsen an important subgroup or edge case. For broader evaluation guidance, see the machine learning model evaluation guide.

Common mistakes

  1. Preprocessing before splitting. Calculating global means, scaling parameters, feature-selection scores, or text vocabularies before the split allows evaluation data to influence training.
  2. Using target encoding without strict validation. Replacing a category with its average target can be useful, but it must be calculated without exposing the row’s own label and should respect groups or time.
  3. Creating features from post-outcome fields. A resolved status, final invoice amount, or later customer action may look predictive while being unavailable at decision time.
  4. Encoding identifiers as ordinary categories. Unique IDs often encourage memorization rather than generalization. Test whether they represent a stable, meaningful grouping.
  5. Ignoring production drift. A category set, numeric range, or text vocabulary can change after deployment. Log input distributions and transformation failures.
  6. Adding complexity without a baseline. Begin with a simple, documented feature set. More elaborate transformations are easier to judge when there is a reliable baseline.

Feature engineering is part of the production machine learning workflow, not just a notebook activity. Once a model is deployed, monitor input quality and distribution changes alongside model performance. The model monitoring guide covers the maintenance questions that follow deployment.

When to revisit

Revisit the feature set whenever the data-generating process changes, before a seasonal planning cycle, or when the workflow or preprocessing libraries change. A new product, pricing rule, collection method, form field, or business definition can make an existing feature misleading even if the code still runs.

Use this short review before each significant model iteration:

  1. Reconfirm the target definition and prediction timestamp.
  2. Compare current input distributions with the training snapshot.
  3. Check missingness, new categories, outliers, duplicate patterns, and stale features.
  4. Re-run leakage checks and confirm the validation split still reflects use in practice.
  5. Compare the candidate against a documented baseline using the same metrics and error analysis.
  6. Update the feature log, pipeline tests, and deployment documentation.

When a library release changes encoder, imputer, or pipeline behavior, run preprocessing tests on representative training, validation, and production-like rows before retraining. For a complete project structure, combine this checklist with an end-to-end machine learning project checklist. The goal is not to engineer more features indefinitely; it is to maintain features that are valid, reproducible, measurable, and available when the model needs them.

Related Topics

#feature-engineering#data-preprocessing#scikit-learn#machine-learning#data-science
S

Skilling.pro Editorial Team

AI and Machine Learning Education Editors

Senior editor and content strategist. Writing about technology, design, and the future of digital media. Follow along for deep dives into the industry's moving parts.