跳到主要内容

2 A Brief Overview of AI

A history recap of AI​

What Is AI? — History Recap (Key Points)​

1. Foundations of AI (1950s–1960s)​

  • 1950 – Alan Turing
    • Paper: “Computing Machinery and Intelligence”
    • Introduced the Turing Test and the question “Can machines think?”
  • 1956 – Dartmouth Workshop
    • The term Artificial Intelligence was coined
    • Key figures: John McCarthy, Claude Shannon, Marvin Minsky
  • Early Symbolic AI
    • Intelligence described using logic, rules, and symbols
    • Chess and language viewed as ideal testbeds

2. Symbolic AI & Expert Systems (1970s–1980s)​

  • John McCarthy
    • Invented LISP (1950s), the main AI language of the time
    • AI based on formal logic and rule-based reasoning
  • Expert Systems
    • Large collections of human-written if–then rules
    • Specialized hardware (LISP machines)
    • Created a billion-dollar industry
  • First AI Spring
    • Strong optimism about rule-based intelligence

3. AI Winters & Limitations​

  • First AI Winter (1970s)
    • Early neural networks (perceptrons) failed on complex tasks
  • Second AI Winter (late 1980s–early 1990s)
    • Expert systems proved brittle and expensive
    • Collapse of the LISP machine market
    • Often described as the “end of pure symbolism”
  • Core Problem
    • Hard to precisely describe perception and patterns using rules

4. Statistical Machine Learning (1990s–2000s)​

  • Shift from rules to data-driven learning
  • Models learned from data but relied on human-designed features
  • Key advances:
    • Boosting (1990)
    • Support Vector Machines (1993)
    • Random Forests (1995)
  • Tom Mitchell’s definition
    • A program learns from experience E on tasks T if performance improves

5. Probabilistic & Bayesian AI​

  • Judea Pearl (Turing Award 2011)
    • Introduced Bayesian networks and causal reasoning
  • Unified:
    • Probability
    • Human knowledge (priors)
    • Machine learning
  • Marked the rise of probabilistic graphical models

6. The Deep Learning Revolution (2012– )​

2012: The Magic Year​

  • AlexNet (Krizhevsky, Sutskever, Hinton)

    • Won ImageNet with a huge margin
    • First modern SOTA model using deep neural networks
    • End-to-end learning from raw data using GPUs

  • Humans design architectures, not features

Language Domain​

  • 2003 – Neural Language Model (Bengio)
    • First neural approach to language modeling
  • 2014 – Seq2Seq (Google)
    • First end-to-end neural machine translation model
  • 2016 – Google Neural Machine Translation
  • 2017 – Transformer
    • Became the dominant architecture for sequence modeling
    • Achieved SOTA across NLP, vision, and beyond

Reinforcement Learning​

Motivation: Beyond Supervised Learning​

  • Many intelligent tasks require a sequence of decisions, not a single prediction.
  • Agents must interact with an environment and adapt based on feedback.
  • This setting goes beyond standard machine learning → Reinforcement Learning.

Core Framework of Reinforcement Learning​

  • Agent: the decision-maker, defined by a policy (\pi(a|s)) or (\mu(s;\theta))

  • Environment: the external system the agent interacts with

  • State ((s_t)): the current situation of the environment

  • Action ((a_t)): the choice made by the agent

  • Reward ((r_t)): feedback signal evaluating the action

  • Learning paradigm: trial and error

  • Suitable for temporal decision-making problems

Foundation of modern RL​

  • Policy Gradient Methods for Reinforcement Learning with Function Approximation
  • Richard S. Sutton, David McAllester, Satinder Singh, Yishay Mansour
  • NIPS 1999