My midterm project at MLZoomcamp led by Alexey Grigorov for DataTalksClub

 Predicting Patient No-Shows: A Data-Driven Approach


Hospital no-shows significantly disrupt healthcare systems, wasting resources and delaying care for those in need. My midterm project for the MLZoomcamp, led by Alexey Grigorev and hosted by DataTalksClub, tackles this challenge using machine learning to predict no-show probabilities for appointments in Brazilian hospitals. Here's how I approached the problem:

The Challenge

The dataset, sourced from Kaggle, includes over 110,000 appointments and diverse features such as patient demographics, appointment details, and medical history. However, achieving reliable predictions is complex due to:

  • Imbalanced Data: About 80% of appointments were attended, while 20% were no-shows.
  • Dependence on Feature Engineering: Key predictors like patient history (previous/missed appointments) were engineered from the raw data.
  • Bias Mitigation: Socioeconomic factors, such as neighborhood, required careful handling to ensure fairness.

The Solution

Through rigorous data cleaning, feature engineering, and hyperparameter tuning, I built and evaluated several models, including Logistic Regression, Random Forest, and XGBoost. Here’s why XGBoost emerged as the best:

  • Accuracy: It achieved an AUC of 0.748, outperforming other models in predicting no-shows.
  • Fairness: I ensured the model emphasized medical and appointment-specific factors over potentially biased features.

Key Insights
  • Lead Time Matters: The days between scheduling and appointment strongly correlate with no-shows.
  • Patient History is Key: Patterns of previous and missed appointments enhance prediction accuracy.
  • Fairness-Driven Engineering: Assessing features for unintended bias was crucial for a balanced model.

Practical Utility

This model can serve as a resource for hospitals to:

  • Identify high-risk appointments early.
  • Optimize resources by managing no-show probabilities.
  • Enhance patient care with targeted reminders or interventions.


My journey through this project reflects the potential of machine learning to solve real-world problems. If you're intrigued, feel free to explore the code and methodology in my GitHub repository.










Comments

Post a Comment

Popular posts from this blog

Logistic Regression: A walkthrough by Alexey Grigorev

Logistic Regression is one of the foundational algorithms for classification tasks, and Alexey Grigorev at DataTalks.Club provides an insightful approach to understanding it in his courses, such as the Machine Learning Zoomcamp. Alexey Grigorev provides a clear and practical explanation of Logistic Regression, helping learners understand its application and limitations in real-world scenarios. In his teaching, Alexey emphasizes the simplicity and interpretability of Logistic Regression. The model predicts the probability that a given input belongs to a particular class, which makes it especially useful in binary classification problems. For example, it’s widely applied in fields like customer churn prediction, fraud detection, and medical diagnoses. Logistic Regression works by modeling the relationship between input features and the probability of a binary outcome using a sigmoid function. This makes the predictions constrained between 0 and 1. Alexey often stresses the importance of ...
Read more

Linear Regression: A Deep Dive with Alexey Grigorev

  Linear Regression: A Deep Dive with Alexey Grigorev Linear regression is a cornerstone of machine learning, and Alexey Grigorev's teachings at Data Talks Club provide a comprehensive and insightful exploration of this fundamental algorithm. Key Concepts Covered by Alexey Grigorev: Simple Linear Regression: Understanding the relationship between a single independent variable and a dependent variable. Multiple Linear Regression: Modeling relationships with multiple independent variables. Assumptions: Exploring the underlying assumptions of linear regression, such as linearity, independence, normality, homoscedasticity, and no multicollinearity. Model Evaluation: Learning how to evaluate the performance of a linear regression model using metrics like R-squared, mean squared error (MSE), and root mean squared error (RMSE). Regularization: Understanding techniques like Ridge and Lasso regression to prevent overfitting and improve model generalization. Feature Engineering: ...
Read more