Thinking About Bioinformatics

As I discussed in a previous post, I want to emulate the admirable clarity and accessibility of Feynman’s Lectures on Physics in my own attempt to write an introductory textbook on information metrics for statistical inference.  Below are my thoughts on how I can apply the lessons that I drew from Feynman in my previous post.

More to the point, I’ve rewritten Chapter 1  What is Inference? based on these lessons.  So now I ask you: is this a genuine improvement?  Note that this is an intro chapter with only the simplest math (some addition and multiplication), so anyone should be able to understand it and critique it!  Please add comments to this post to give your opinion of whether you think the specific changes I outline below improve the chapter, compared with the original version.  I am particularly interested in both whether you think the ideas in my plan are the right direction to pursue, versus whether their actual “reduction to practice” in the new draft chapter works or not.  Above all, tell me how I need to improve my chapter and my writing!

(more…)

Yesterday I talked about applications of potential information to experiment planning, using the example of a robot seeking to discover the principles of genetics from the initial observation of a “mutant” pea plant with white flowers. You can listen to the audio (right click on the audio link, and Save Link As, then listen to the downloaded file using QuickTime player or Real player). I also captured most of the material I wrote on the whiteboard.  Some relevant background material (and detailed exposition of the RoboMendel example) is also available.

Well, I failed to record my seminar audio, but here are some relevant notes for material discussed in the third seminar. This time we discussed the application of information metrics to experiment planning, rather than just model selection. One metric that I emphasized this time is the notion of potential information, which provides a signal for whether the current model needs to be expanded because its fit to the observations is inadequate. The attached material discusses some concrete examples of potential information, for example, for experiment planning.

Here are my slides for my second talk at IMA on Feb. 14. I tried to introduce some problems with typical information metrics as they apply to statistical inference problems. Then I describe empirical information, my preferred information metric for statistical inference. The slides are available as a PDF, and the audio of the talk is also available — you can use either RealPlayer or the Quicktime player to listen to this. To download the audio, right-click on this link and choose Save Link As…

I’ve also posted some background material cut from different chapters of my draft textbook as a PDF.

Here are a couple of items relevant to my Feb. 7 intro session at IMA:

• Chapter 1 draft introduction to probabilistic inference.
• Paper on identification of SNPs from EST chromatogram data.

Let’s examine the principles for a general process of learning, via a scientific example: could we program a robot to make scientific discoveries directly from raw observations? As an example problem, let’s take the discovery of the basic principles of genetics by Gregor Mendel and subsequent researchers.

(more…)

I’m writing a textbook on Information Evolution. Or at least I thought I was — so far it mainly seems to be about the statistical inference side of “information”, as opposed to the “evolution” side. I suspect it will make more sense to make this focus on inference and methodology, and leave the science of how physical systems produce information for a later effort. If you have an interest in the basic issues I’m raising in the posts here, you may want to take a look at the first five draft chapters. That’s where the real meat is.

Does there exist an information metric with truly general utility? If so, a scientist could use it to choose which experiment to do: the best experiment is that one that yields the largest amount of information about the scientist’s question of interest (or, over the long-term, the highest information rate per unit time / expense). Indeed, if the metric were truly general, the scientist could use it to decide which research question is “most interesting” (again, compute the expected information yield for the different research directions). Actually, if such an information metric existed, the “scientist” could just be a robot, because all that is required is the ability to calculate this metric for different possible experiments (observations). This wouldn’t be artificial intelligence in the traditional sense of that field, but instead just a big statistical number-crunching computation. In a way, scientific computing at its dullest.
(more…)

Welcome to Thinking About Bioinformatics! In this post, I’ll try to explain my goals for this blog, and the kind of topics I plan on writing about. For some time I’ve been very interested in the nature of information, and processes that produce information. Meanwhile, my work in bioinformatics has paid the bills. I’m going to use this blog as a place to try to start discussion of information producing processes with other people who are interested in these questions. So please feel free to post a comment, send email to me, or link your own blog material to these posts. Here are some of the kinds of topics I hope to write about:

• the general information metric hypothesis: the notion that there is a general measure of information that in some sense is the “answer to all questions” in science, i.e. the best experiment to do is the one that maximizes the information yield and rate of production. There are many interesting arguments both for and against this idea, so in my view this is a great place to put everything we think we know about “information” to a rather searching test. Much of what I’m going to post here focuses on the idea that we can only understand information in Bayesian terms, i.e. as a hidden property of observable variables.
• biology as information and information as biology: there are striking parallels between the population genetic theory of evolution and the theory of statistical inference. For example, the equation for the evolution over time `t` of an asexual haploid population `p_i(t)` under natural selection `W_i^t` is identical to Bayes Law:

`p_i(t)=\frac{p_i(0)W_i^t}{\sum_j p_j(0)W_j^t}`

Here the initial population frequency of a given genotype `p_i(0)` is exactly equivalent to the Bayesian prior probability, the haplotype’s fitness `W_i` is analogous to the likelihood of the observations in Bayes Law, and the population frequencies `p_i(t)` after time `t` are equivalent to the Bayesian posterior probability. So is evolution a big computer doing Bayesian inference on fitness? This is but a small example of a general point: most of the interesting questions about genomes and evolution are most productively understood as questions about information: about statistical inference, about algorithmic complexity etc. And biology probably has things to teach us about statistical inference, as the information-producing process par excellence.

• the theory of powertools: a powertool takes an extraordinary ability and converts it into a rote cycle of steps, rendering it scalable (which to me generally means info-linear, i.e. a simple linear relationship between the amount of information produced, and the amount of work required to produce it). For example, the calculus is a powertool. The most brilliant Greek mathematicians struggled to derive and prove volume equations for various curved solids, which now “any schoolboy” instructed in the calculus can obtain with ease. A powertool is the basis set for a space that “diagonalizes” an entire class of problems into an info-linear representation. Thus the theory of powertools is closely related to the theory of representation.

Anyway, these are my interests, and I’m eager to talk with others who share these interests.

Categorization: for better or worse, my interests and work range from pretty abstract to very concrete / practical (e.g. Pygr), so I’m going to try to assign each post to separate categories so people can pick out the parts they’re interested in from the rest.