This Year's Theme: Mathematical Modeling of Information, Communication, and Intelligence
Subjects: Information Theory, Neural-/Inter-Networking, Wireless Communication, Bio-Signaling/-Information, Basic Quantum Info.
Text: All the lectures will be mainly based on my developing lecture notes.
Main Tools: Linear Algebra, Calculus, Differential Equations, Probability, Discrete Mathematics, Basic Physics and Biology.
Covered Topics (dynamically updated, as of April 20 ) (best viewed in Firefox Browser/Netscape for mathematical formulae):
- An Immune Cell Chasing a Bacterium: "seeing" by smelling; signaling, and processing. dx/dt=-&alpha dC/dx or -&alpha &nabla C(x).
- Smelling/Olfaction (2004 Nobel Prize): odorant signals, threshold, neuron firing, firing patterns. y=ground( (x1+ ... +xn)/&sigma ).
- DNA and Genes: Nucleotide signals ATCG, Phylogenetic distance, Markov single-site mutation. M=[ pbase i to base j ]4x4, Mn.
- Genetic Regulation: Initiating Signals &rArr Genes &rArr Proteins &rArr Structures (Head/Limbs): p(x)=P(G(s(x))), dp/ds=dP/dg &bull dG/ds (Chain Rule for the analysis of signal-protein sensitivity).
- Single Bit and Entropy: One bit (simplest information), entropy, concavity, fractional bits, Bernoulli random variable (simplest r.v.): Bernoullli(p), S(p)=-p log(p) - (1-p)log(1-p), dS/dp, d2 S/dp2.
- 2-Bit Systems and
Mutual Information: T={0,1}: Tom's light house (on or off) on Island Summer; A={0,1}: Alice's light house on
Island Breeze. We make observation each night at 10:00pm from Island Banana. How can we detect whether Tom and Alice know
each other, or are communicating and coordinating on their lights:: the extraordinary (!) parallel between:
Mass Weighing: |T &cap A|=|T| +|A| - |T U A|, and Mutual Information: I(T, A)= S(T) + S(A) - S(T, A). -
General 2-Event Systems: X, Y, state space &Omega, chances and probabilities, marginals from the joint, mutual information:
S(X)= -p1log p1 - ... - pN log p N ; max S(X)=log |&Omega| (by Multivariate Calculus ); I(X, Y) = S(X) + S(Y) - S(X, Y).
-
Information Transform & Communication:
transform method, linear transform, Haar transform, analysis, synthesis, aliasing:
x (input signal) &rArr [analysis transform T] &rArr c (transmitted) &rArr ... &rArr C (received) &rArr [synthesis transform S] &rArr X (recovered signal)
Modeling Wireless Phones: x (ur voice) &rArr [T by ur cellphone] &rArr c (electromagnetic waves) &rArr ... (silently flying waves) ...
&rArr C (received via antenna) &rArr [S by your friend's cellphone] &rArr X (your voice as heard): non-aliasing iff X=x.
- [Spring Break]
Recommended Midterm Project: Modelling the Communication and Collaboration of Ants.
- Waves, Messages, Phones, and Radio Stations: sine waves, cosine waves,
the meaning of "FM 91.1 MHz" (NPR radio station in Minneapolis), the meaning of your cell phone number, forward transform,
and backward retrieval, orthogonality, and ``perpendicular" cell phones: w(t)=a0 + a1 cos(t)+ ... + a10 cos(10 t),
messages (a0, a1, ..., a10) (11 cellphone users) &rArr waveform w(t) flying in the air &rArr extracted messages: a0, a1, ..., a10.
- Patterns, Features, Driver Licences, and Decision Making:
A pattern is a specific organization
of features (or smaller sub-patterns, which leads to an iterative definition); A feature is the reading or output
from a sensor measurement (e.g., weight by scales, color by photoreceptors, height by rulers...) [mathematically,
often given by a projection (onto a linear component) via inner products];
likelihood functions, maximum likelihood decision; tuna or salmon?
- Bayesian Decision, a (Scientifically) Good Person, and a Bayesian Robot: Bayesian view of a "good" or "ideal" person:
(1) open, non-resistant to incoming data; (2) generally knowledgable, with good knowledge on the state of the world (prior model); and (3) specifically knowledgable, knowing well the phenomena/symptons associated with each specific state (data model).
One can only make good decisions by combining the sensed data, the prior model, and the data model via:
bet on (or choose to believe) S=s (a specific state or pattern) which maximizes the posterior chance: Our working example: Prob(Fish=tuna) and Prob(Fish=salmon) are known (prior knowledge of the river, learned from ancesters)
Prob(world state S=s | Sensed Data)=Prob( Sensed Data | S=s) Prob(S=s)/a constant, or in English,
One must combine the predictive data model (from State to phenomenological Data) and the prior knowledge on the world State.
Prob(weight | tuna) and Prob(weight | salmon) are two predictive data model. Storing them in its memory, a Bayesian robot can then report if a new fish is salmon or tuna, after putting the adult fish in its palm (with a built-in scale) and reading the weight.
(Disclaimer: A scientifically good person is unnecessarily a good person socially, religiously, or in terms of decency/integrity.) - Learning, Training, Model Selection, and Parametric Data Fitting:
Learning is crucial for both H.I. (human intelligence) and A.I. (artificial intelligence);
In the above Bayesian framework, to learn is to establish both the Prior Model (on the states of the world), and the
Data Model (on the symptons of each state); Mathematically, LEARNING = Data fitting or function fitting:
parametric modeling and learning: Cauchy, Gaussian N(m, &sigma2), polynomial least-square fitting, and linear regression.
Monte-Carlo estimator for integration( f(w)p(w) dw), and applications in parameter learning of m and &sigma2 for Gaussian.To ponder: how does a typical baby learn which tricks (crying, giggling, kicking...) work the best to get milk from its mom?