Welcome to the Data Science Learning Notebooks repository! This project serves as a comprehensive resource for students, practitioners, and enthusiasts looking to learn and deepen their understanding of data science concepts. The repository features a collection of Jupyter notebooks covering both fundamental and advanced topics in data science, blending theoretical insights with practical applications.
Why Data Science?
Data Science is the art of transforming raw data into actionable insights and data-driven products. These products empower decision-makers by providing clear, actionable information while abstracting away the complexities of underlying data and analytics. Examples of such products include:
- Buy/Sell Strategies for financial instruments
- Product Optimization Actions to enhance manufacturing yield or improve marketing efforts
- Logistics Optimization Solutions to streamline operations
The foundation of effective data science lies in understanding the data you have and what it reveals inductively.
Questions like the following can be addressed with well-designed data products:
- Which of my products should I promote to maximize profit?
- How can I improve compliance programs while reducing costs?
- What process changes can help build a better product?
By learning data science, you gain the ability to create algorithmic products and services, such as recommendation systems, customer engagement optimizers, trend detectors, and many others. These systems enhance business capabilities and drive innovation.
The discipline of computer science, which forms the bedrock of data science, has evolved significantly since its inception in the 1960s. Early focuses included programming languages, compilers, operating systems, and foundational mathematical theories. By the 1970s, algorithms and their practical applications became a central focus. Over the decades, this emphasis shifted to reflect the convergence of computing, communication technologies, and the explosion of data generation.
Today, the focus is on extracting actionable insights from massive datasets rather than solving strictly defined problems. This shift highlights the growing importance of:
Probability and Statistics Numerical Methods Machine Learning and AI Algorithms
These areas enable us to tackle challenges presented by large-scale data, internet applications, and social networks.
What This Repository Offers: This repository is designed to bridge the gap between theoretical understanding and practical implementation. It provides:
- Basic Concepts: Introduction to statistical analysis, data manipulation, and foundational algorithms.
- Advanced Techniques: Topics such as machine learning, deep learning, and probabilistic modeling.
- Practical Applications: Case studies on building recommendation systems, optimizing marketing strategies, and more.
- Mathematical Rigor: Exploration of mathematical and computational techniques that underpin data products. Enduring Concepts: A focus on timeless ideas that have shaped data science and continue to be relevant. Use Cases
The notebooks in this repository can help you:
- Learn and Practice Core Data Science Concepts – Ideal for beginners starting their journey or practitioners looking to reinforce their knowledge.
- Explore Advanced Techniques – Dive deep into the algorithms and ideas behind creating powerful data products.
- Develop Real-World Skills – Build actionable insights from data using statistical analysis, machine learning, and computational modeling.
By using these resources, you will gain the ability to:
- Understand and explore data to uncover actionable insights.
- Build data products that provide meaningful information for decision-making.
- Develop a profound understanding of the algorithms and ideas that drive data science forward.
In practice these ideas, skill and knowledge can be applied to develop new business capabilities. For example the ideas can be used to develop Algorithmic products and services like recommendations systems, client engagement bandits, style preference classification, size matching, fashion design systems, logistics optimizers, seasonal trend detection and so on.
This repository is a living resource, and we welcome contributions from the community to ensure it stays up-to-date with the latest developments in data science. Whether you’re a student, educator, or seasoned practitioner, we hope these notebooks will serve as a valuable asset in your learning journey!
https://computationalthinking.mit.edu/Fall24/
Jeremy Kun etc
References:
https://github.com/asjad99/Algorithms-for-data-products CS168, advanced analytics in Spark Data science foundations book by CMU
Welcome to the Fundamental Concepts in Data Science Repo!
This Repo is a collection of Jupyter notebooks aimed at teaching key concepts in data science, ranging from foundational mathematics to practical data analysis techniques. Each notebook is designed to be a standalone learning resource, complete with explanations, examples, and exercises to help you get hands-on with each topic.
Below is an index of the notebooks included in this repository. Links to the individual notebooks will be added soon.
| Topic | Description | Link |
|---|---|---|
| 1. Exploratory Data Analysis (EDA) | EDA Notebook 1: Introduction to EDA and basic data summary techniques EDA Notebook 2: Understanding distributions, central tendency, and variability EDA Notebook 3: Univariate and multivariate relationships in data | [Link Placeholder] |
| 2. Data Munging | Data Cleaning and Preprocessing: Handling missing data, data transformation, and feature engineering Data Wrangling: Merging, reshaping, and dealing with categorical data | [Link Placeholder] |
| 3. Linear Algebra | Vectors and Matrices: Concepts of vectors, operations on matrices, and matrix factorizations Applications in Data Science: How linear algebra is used in machine learning models | [Link Placeholder] |
| 4. Data Visualization Techniques | Basic Plotting: Introduction to Matplotlib and Seaborn for visualizing data Advanced Visualization: Creating interactive visualizations and dashboards | [Link Placeholder] |
| 5. Statistics | Descriptive Statistics: Measures of central tendency and variability Inferential Statistics: Hypothesis testing, confidence intervals, and p-values | [Link Placeholder] |
| 6. Experimental Design and Analysis | Experimental Design: Concepts of controlled experiments, A/B testing, and sample size determination Analysis Techniques: Methods for analyzing experimental results | [Link Placeholder] |
| 7. Dimensionality Reduction | PCA and t-SNE: Introduction to Principal Component Analysis and t-Distributed Stochastic Neighbor Embedding Feature Selection: Techniques for selecting important features in a dataset | [Link Placeholder] |
| 8. Clustering | K-Means and Hierarchical Clustering: Understanding unsupervised learning and cluster formation Clustering Evaluation: Techniques to evaluate clustering effectiveness | [Link Placeholder] |
| 9. Graphs | Introduction to Graphs: Understanding nodes, edges, and types of graphs Network Analysis: Concepts like centrality, shortest path, and community detection | [Link Placeholder] |
| 10. Numerical Optimization | Optimization Basics: Gradient descent, learning rates, and optimization algorithms Applications: Optimization techniques in machine learning models | [Link Placeholder] |
| 11. Storytelling with Data (Our World in Data) | Data Storytelling Techniques: How to effectively communicate insights using real-world datasets Examples from Our World in Data: Exploring global datasets to tell compelling stories | [Link Placeholder] |
Contributions are welcome! If you would like to add new notebooks, suggest changes, or fix any issues, please feel free to submit a pull request.
Advanced Algorithms: https://github.com/asjad99/Algorithms-for-data-products Everyday DS Tools: https://github.com/asjad99/Data-Science-Tools Case Stuides/Applications: https://github.com/asjad99/Data-Science-Applications
- Inferring Concept Drift Without Labeled Data
- Session-based Recommender Systems
- Causality for Machine Learning
- Interpretability
- Deep Learning for Anomaly Detection
- Learning with Limited Labeled Data
- Federated Learning
- Probabilistic Methods for Realtime Streams
- Distributed process event data mining using federated learning
- Sequential User Session Data for build recommender system using LSTMs
- Anomaly detection for predictive Maintenance of Turbofan engine / aviation
References:
- Fast Forwards Labs
“We’re entering a new world in which data may be more important than software.” — Tim O’Reilly
“You can best learn data mining and data science by doing, so start analyzing data as soon as you can! However, don’t forget to learn the theory, since you need a good statistical and machine learning foundation to understand what you are doing and to find real nuggets of value in the noise of big data.”
“The best way to learn data science is to do data science.” — Chanin Nantasenamat
“The era of Data Technology is here and it will surpass the Information Technology era. The DT era is about transparency, sharing of information and enabling others. Alibaba is excited about the possibilities of the DT era and how it can bring value to society.” — Jack Ma
“Many believe that Big Data is over-hyped, but seeing the fantastic use cases popping up around the globe I would say Big Data is under-hyped! In the coming years, Big Data will revolutionize every industry unlike we have seen before!”
“I like to think of data as the new soil, Get in and get your hands dirty.” — David McCandless
“Data Scientist (n.): Person who is better at statistics than any software engineer and better at software engineering than any statistician.” — Josh Wills
“Despite an awful lot of marketing hype, big data are here to stay and big data analytics (i.e. data science and statistics) will remain aids to human thinking and not replacements for it!” — Diego Kuonen
“For me, data science is a mix of three things: quantitative analysis (for the rigor necessary to understand your data), programming (so that you can process your data and act on your insights), and storytelling (to help others understand what the data means).” — Edwin Chen