Course Website

The term no one really defined

• What is data science?

Data science is the discipline of making data useful. Ok…so what is it?

Three tracks of data science

• Engineering (infrastructure and production): the process of making everything else possible

• Analysis: the process of turning raw information into insights in a fast way

• Modeling/Inference: the process of diving deeper into the data to discover the pattern we don’t easily see

(It is a group work from https://github.com/brohrer/academic_advisory/blob/master/authors.md !)

Engineering

1. Data environment: data storage, Kafka platform, Hadoop and Spark cluster etc.

2. Data management: parsing the logs, web scraping, API queries, and interrogating data streams.

3. Production: integrate model and analysis into the production system

Data Pipeline

Analysis

1. Domain knowledge

2. Exploratory analysis

3. Story telling

Modeling

1. Supervised learning

2. Unsupervised learning

3. Customized model development

Some confusions and more to come

• Data infrastructure engineer v.s. Data engineer
• Data engineer v.s. BI engineer
• BI engineer v.s. Data analyst
• Data analyst v.s. Data scientist
• …

General Process of Modeling/Analytics

Three tracks of data science

Three tracks of data science

Three tracks of data science

Data Science Curriculum Roadmap

What can (should) data science do?

• Make human better human by alleviating bounded rationality and minimize politics/emotion (rather than make machine more like human)

• Strive for the “democratization” of data as legally possible: empower everyone in the organization to acquire, process, and leverage data in a timely and efficient fashion

Data Science Hierarchy of Needs

Data Science Types v.s Needs

Data Science Types v.s Needs

Data Science Types v.s Needs

Data Science Types v.s Needs

Types of Questions (Modeling/Analytics)

• Comparison:
  • Are males more inclined to buy our products than females?
  • Are there any differences in customer satisfaction in different business districts?
  • Do soybean carrying a particular gene have higher oil content?
• Description:
  • Is the distribution of annual income normal?
  • Are there outliers?
  • What are the means of different customer segments?
• Clustering:
  • Which customers have similar product preference?
  • Which printer performs similar pattern to the broken ones?
  • How many different themes are there in the corpus?

Types of Questions (Modeling/Analytics)

• Classification:
  • Who is more likely to buy our product?
  • Is the borrower going to pay back?
  • Is it a spam?
• Regression:
  • What will be the temperature tomorrow?
  • What is the projected net income for the next season?
  • How much inventory should we have?

Where does data science belong in your organization?

• An autonomous unit parallel to engineering
• A strategic differentiator for the organization
• The head of data science unit should ideally report into the CEO or at least report into someone who understands data strategy and is willing to invest to give it what it needs.
• Pros:
  • autonomy
  • well positioned to tackle whatever problems it deems most valuable
  • knowledge share and grow
  • attract/retain talent
• Cons: risk of marginalization

Where does data science belong in your organization?

• Bring in talented people and farms them out to the rest of the company. There’s still a head of data science, but he/she is mostly a hiring manager and coach.
• Give up autonomy to ensure utility
• Pros:
  • closer to applications
  • flexibility
• Cons:
  • hard to attract/retain talent
  • their embedded team leads don’t feel responsible for their growth and happiness, while their managers won’t feel directly vested in their work
  • data scientists are second-class citizens everywhere

Where does data science belong in your organization?

• No separate data science team; each team hires and manages its own data scientists.
• Pros:
  • optimizes for organizational alignment;
• Cons:
  • dilutes the identity of data science;
  • hurt data scientist’s career growth;
  • sacrifice the flexibility of the embedded model

Some links

• Part of the deep learning slides are based on Andrew Ng’s course: Deep Learning Specialization: Super awesome!

• Industry recommendations for academic data science programs

• Awesome-Data-Science-Materials

• Online books:
  • The Elements of Statistical Learning
  • An Introduction to Statistical Learning
  • Introduction to Data Science(still writing)
• Hard copy books:
  • Applied Predictive Modeling
  • R for Marketing Research and Analytics