An Introduction to Natural Computation
Dana H. Ballard, 1997

Chapter 1. Natural Computation

1.1. Introduction

1.2. The Brain

1.2.1. Subsystems

1.2.2. Maps

1.2.3. Neurons

1.3. Computational Theory

1.4. Elements of Natural Computation

1.4.1. Minimum Description Length

1.4.2. Learning

1.4.3. Architectures

1.5. Overview

1.5.1. Core Concepts

1.5.2. Learning to React: Memories

1.5.3. Learning During a Lifetime: Programs

1.5.4. Learning Across Generations: Architectures

1.6. The Grand Challenge

Part I. Core Concepts

Chapter 2. Fitness

2.1. Introduction

2.2. Bayes' Rule

2.3. Probability Distributions

2.3.1. Discrete Distributions

2.3.2. Continuous Distributions

2.4. Information Theory

2.4.1. Information Content and Channel Capacity

2.4.2. Entropy

2.4.3. Reversible Codes

2.5. Classification

2.6. Minimum Description Length

Appendix: Laws of Probability

Chapter 3. Programs

3.1. Introduction

3.2. Heuristic Search

3.2.1. The Eight-Puzzle

3.3. Two-Person Games

3.3.1. Minimax

3.3.2. Alpha and Beta Cutoffs

3.4. Biological State Spaces

Chapter 4. Data

4.1. Data Compression

4.2. Coordinate Systems

4.3. Eigenvalues and Eigenvectors

4.3.1. Eigenvalues of Positive Matrices

4.4. Random Vectors

4.4.1. Normal Distribution

4.4.2. Eigenvalues and Eigenvectors of the Covariance Matrix

4.5. High-Dimensional Spaces

4.6. Clustering

Appendix: Linear Algebra Review

Chapter 5. Dynamics

5.1. Overview

5.2. Linear Systems

5.2.1. The General Case

5.2.2. Intuitive Meaning of Eigenvalues and Eigenvectors

5.3. Nonlinear Systems

5.3.1. Linearizing a Nonlinear System

5.3.2. Lyapunov Stability

Appendix: Taylor Series

Chapter 6. Optimaization

6.1. Introduction

6.2. Minimization Algorithms

6.3. The Method of Lagrange Multipliers

6.4. Optimal Control

6.4.1. The Euler-Lagrange Method

6.4.2. Dynamic Programming

Part II. Memories

Chapter 7. Content-Addressable Memory

7.1. Introduction

7.2. Hopfield Memories

7.2.1. Stability

7.2.2. Lyapunov Stability

7.3. Kanerva Memories

7.3.1. Implementation

7.3.2. Performance of Kanerva Memories

7.3.3. Implementations of Kanerva Memories

7.4. Radial Basis Functions

7.5. Kalman Filtering

Chapter 8. Supervised Learning

8.1. Introduction

8.2. Perceptions

8.3. Continuous Activation Functions

8.3.1. Unpacking the Notation

8.3.2. Generating the Solution

8.4. Recurrent Networks

8.5. Minimum Description Length

8.6. The Activation Function

8.6.1. Maximum Likelihood with Gaussian Errors

8.6.2. Error Functions

Chapter 9. Unsupervised Learning

9.1. Introduction

9.2. Principle Components

9.3. Competitive Learning

9.4. Topological Constraints

9.4.1. The Traveling Salesman Example

9.4.2. Natural Topologies

9.5. Supervised Competitive Learning

9.6. Multimodal Data

9.6.1. Initial Labelling Algorithm

9.6.2. Minimizing Disagreement

9.7. Independent Components

Part III. Programs

Chapter 10. Markov Models

10.1. Introduction

10.2. Markov Models

10.2.1. Regular Chains

10.2.2. Nonregular Chains

10.3. Hidden Markov Models

10.3.1. Formal Definitions

10.3.2. Three Principle Problems

10.3.3. The Probability of an Observation Sequence

10.3.4. Most Probable Spaces

10.3.5. Improving the Model

Chapter 11. Reinforcement Learning

11.1. Introduction

11.2. Markov Decision Process

11.3. The Core Idea: Policy Improvement

11.4. Q-Learning

11.5. Temporal-Difference Learning

11.6. Learning with a Teacher

11.7. Partially Observable MDPs

11.7.1. Avoiding Bad States

11.7.2. Learning State Information from Temporal Sequences

11.7.3. Distinguishing the Value of States

11.8. Summary

Part IV. Systems

Chapter 12. Genetic Algorithms

12.1. Introduction

12.1.1. Genetic Operators

12.1.2. An Example

12.2. Schemata

12.2.1. Schemata Theorem

12.2.2. The Bandit Problem

12.3. Determining Fitness

12.3.1. Racing for Fitness

12.3.2. Coevolution of Parasites

Chapter 13. Genetic Programming

13.1. Introduction

13.2. Genetic Operators for Programs

13.3. Genetic Programming

13.4. Analysis

13.5. Modules

13.5.1. Testing for a Module Function

13.5.2. When to Diversify

13.6. Summary

Chapter 14. Summary

14.1. Learning to React: Memories

14.2. Learning During a Lifetime: Programs

14.3. Learning Across Generations: Systems

14.4. The Grand Challenge Revisited


Top of Page | INC Opinion | Sort by Topic | Sort by Title | Sort by Author