More to the point, I’ve rewritten Chapter 1  What is Inference? based on these lessons.  So now I ask you: is this a genuine improvement?  Note that this is an intro chapter with only the simplest math (some addition and multiplication), so anyone should be able to understand it and critique it!  Please add comments to this post to give your opinion of whether you think the specific changes I outline below improve the chapter, compared with the original version.  I am particularly interested in both whether you think the ideas in my plan are the right direction to pursue, versus whether their actual “reduction to practice” in the new draft chapter works or not.  Above all, tell me how I need to improve my chapter and my writing!

 How can I apply Feynman’s lessons to my own writing?

• Short, self-contained chapters following Feynman’s question and answer model. I’ve struggled a bit over where to put some topics in my current “whole subject” chapters. I think these topics (e.g. Law of Large Numbers; basic info theory relations, etc.) will work better as self-contained chapters, which also make the precise order of topics less important. This seems like a useful goal. It will force me to relate each topic to my fundamental questions. It makes the material more accessible, like John Baez’s introductions in each This Week’s Finds. Reducing the stress on exact linear order will make it much easier to use this material in multiple formats, and to get feedback from other people. (I’ve never been able to get feedback on more than one chapter, from anyone). Finally, it’s probably an egomaniacal delusion to stuff a whole subject entitled “Inference” into one chapter. This layout pretends that each subject can be treated separately. A larger number of short chapters may actually work better: while each chapter is self-contained, it will discuss fundamental questions that connect it to many other chapters.

• I can rework much of my material into dialog form. For example, when I introduce the Monty Hall problem, I say there are two points of view on Monty’s “twist”: it doesn’t make any difference: some people find the whole question odd… vs. it makes a difference: some people make the argument that…. This cries out to become an actual dialog (with named characters, quotes, exclamation points!).

• I definitely have a mild case of math-envy, and go out of my way to show that I can drop some equations where relevant. The problem is, this turns into a conventional authority / soporific effect. I’m not sure how to strike the right balance. I guess you just ask yourself what’s truly needed for learning the material best. For example, my little section on rearrangement rules seems more “optional” than “needed”. 
• I need to define for my reader a “science of information” that is not just math. The subject matter (abstract information; statistical inference) implies that it is pure math, but I’m not sure that’s right (especially for me!). The whole point of my approach is to redefine information as empirical. This is not just a methodological detail. First and foremost, it’s my personality. I’m a scientist in outlook, not a mathematician. I can develop these ideas scientifically (i.e. “thought experiments” or computational experiments, or using math as a model just as people do in physics or any other science). It would be better for me to focus this text on my strengths, taking full advantage of the scientific approach that I can contribute, rather than allowing this to become a watered-down math textbook. What are the elements of my “science of information”?

  • the key principle of operational definitions

  • the fundamental distinction between observable vs. hidden states as the foundation for all inference.

  • correspondingly, defining all information as empirical, i.e. measuring our ability to predict observables.

  • consequently, the focus on being able to compute: first, numerical models of actual data; second, measurements of information yield.

• An essential corollary is that I must articulate the fundamental questions of this new science. Each chapter introduction will raise these questions in dialog form, showing that our thinking about basic things is confused and incomplete. This will be a huge improvement over my existing introductions, which tend to be pompous, empty statements of grand intentions. Actually, it would do me a lot of good to have clear statements of these fundamental questions. It will probably turn out that I am just as confused as everyone else, even about how to articulate what the key questions are. Let’s try a few obvious questions:

  • is there a quantitative, scientific theory of information? Can “information” be measured in a way that solves problems generally?
  • how can we establish a clear framework of operational definitions for the phenomena that matter in this field? E.g. “observable” vs. “hidden”; “prediction” in the absence of an “objective observer” etc.
  • what “counts” as information? This has a number of confusing aspects (randomness vs. uniformity; “hidden” information; mutual information; definition of “prediction”), and is the question that empirical information answers.
  • how can we do experiment planning (before we know the result!)?

  • what are the requirements for sustainable (unbounded) information production? What is the difference between biological evolution and, say, clouds forming in the sky (or geological formations underground)? What are the requirements for scalable information production, i.e. a powertool?

  • Can this be made a simple, general procedure for discovering any information structure, or solving any problem?

Object-relational design makes modularity even more difficult than usual. It illustrates a general problem: when you try to combine two different behaviors (e.g. a local Python object and a back-end database) into one object, all sorts of confusion can ensue.

My painful experience with annotation objects exemplified this problem. At first it seemed obvious that you want to have combined access to the sequence object and the annotation attributes. But this quickly turned into a nightmare: since there was no way for a user to indicate whether he wants to deal with the object as a sequence or as an annotation, the object constantly has to guess which behavior is desired. This quickly leads to nasty bugs in every case where the guessing policy fails… The solution was to separate the two behaviors absolutely, by providing a sequence attribute on the annotation object.

__setattr__ Considered Harmful

These abstract issues turn painfully concrete when you write __getattr__ and __setattr__ methods. These methods do not “play nice with others”, to put it mildly. Any time you write a getattr you create a modularity problem, because all attribute requests will come to this one method. If you later use multiple inheritance (i.e. “combine two behaviors in one object” as above) you will be obligated to write a new layer of getattr to act as traffic cop for all the layers of getattr below it. This does not scale. Complexity begins to grow in a self-reinforcing way. For example, you’ll immediately start to get infinite loops (e.g. during unpickling, the unpickler will request attributes like reduce and setstate that may not exist, so getattr gets called, it requests some attribute that doesn’t exist because the object hasn’t even been initialized yet, so getattr calls itself…). This leads to what I’ll call “protective nonsense”, i.e. your code gets littered with protective tests to block certain vulnerabilities.

This whole approach is error-prone, unscalable and unmodular.

setattr makes this problem about a thousand times worse. Whereas getattr is “the method of last resort” (only gets called if the usual methods fail), setattr is pre-emptive: the very act of adding it to a class re-directs all attribute writes to this one method. So you should consider setattr a form of contamination; it takes over everything that it touches. Note that just the threat of setattr can spread its baleful effect: even if you think a class might someday be subclassed with a setattr, you have to rewrite all your attribute writes to evade it (e.g. by calling setitem on the object’s dict). Note to self: this is ridiculous!

In my experience, the uses that getattr and setattr are applied to are usually very modest, and don’t justify all these disadvantages. The one case that has some is when one object strictly mirrors the attributes from another object.

Tentative Conclusions

• when in doubt, keep behavior separate. The benefits of completely merging two things into a single object may actually not be very great, compared with simply having a standard way for accessing each object from the other.

• use properties (descriptors) instead of getattr and setattr: properties act like normal attributes (and thus are as modular as regular attributes), rather than taking over the class like getattr and setattr do. I suspect we are biased against creating lots of separate properties when we could achieve the same effect with one getattr() method. But in most cases this is a false economy — the same amount of code is required, but is handled once at “compile time” (by decorating a class with a number of properties) instead of repeatedly on every attribute request (in getattr). The former is better. Perhaps more importantly, conflicting property names could be detected automatically (in theory), whereas sorting out potentially conflicting getattr activities is for experts only. It also seems that properties can be added at any time — even after the shadow class already exists — unlike getattr code, which is fixed prior to module import! It’s also interesting to override the property by simply setting a value in the object dictionary, in which case the property get never gets called.

• This pattern should be done in a totally standard way, specifically my current get_shadow_class() function.

• I need a better name for this pattern than “shadow classing”, which doesn’t explain anything. “Instance subclassing”? “subclass binding”? “automatic subclassing and binding”? I think a function name like get_bound_subclass() is much better than get_shadow_class(). So maybe “subclass binding” is the best suggestion. The name should communicate a couple key ideas

  • we create a subclass for each database, in order to bind attributes that are specific to that database.
  • this is done automatically

Case Study: SQLSequence

One object both acts as an interface to a row in an SQL database, and as a Pygr sequence object. That means some attributes are mirrored from the database, whereas others are purely local Python data, used for strictly internal purposes. So now we have a problem: how is the code supposed to know which attributes are purely internal, vs. which should be treated as queries to the database? This is obviously crucial for modularity. The SQL row class code should not have any dependency on knowing anything about what other classes it might get mixed with.

Proposal

• instead of using getattr on the SQL row class, use the shadow class approach and add descriptors (properties) for all the attributes that come from the back-end database. SQLTable already uses the shadow class mechanism, so why not use it to get rid of getattr and setattr?

Comment: the current situation is quite a muddle. SQLTableBase.objclass() is automatically adding ColumnDescriptor properties in place of all actual SQL columns, in line with what I just proposed, but it clashes with what SQLRow expects (to store id attribute locally). I need to clean this up!! Actually, objclass()’s usage of the factory pattern is totally unnecessary. Descriptors can be added to a class at any time; that’s how pygr.Data does it.

Generalizing the Persistence Pattern

Another interesting pattern to look at is the SQLTableBase.new() method. It calls the object constructor first, then inserts into the DB. The idea here is to preserve the usual distinction between construction (init) vs unpickling (setstate). Reading the object from the database is considered to be a form of persistence, so it gets treated using the unpickling route… This makes a lot of sense, to put Python’s built-in support for persistence to work as a systematic solution. The whole reduce, getstate, setstate system is a much more general solution than just for pickling. The obvious implication is that classes like TupleO and SQLRow should be created using the ClassicUnpickler(klass, state) rather than using the usual constructor pattern klass(**state). The converse implication is that we should use getstate to obtain the data to save into the database.

Does this make sense or does it clash with normal pickling usage? On the one hand, it does generalize the idea that you can save an object persistently to work equally well in regular pickling and other applications. On the other hand, this is quite different from the pygr.Data concept of “saving a reference to a persistent object”. That’s a separate issue. Two separate layers:

• standard persistence: extract the dictionary of attributes required for persistent storage.
• persistent ID: look for a persistent ID to store as a reference, instead of actually saving the data (because we know the data is stored somewhere else, and will always be retrievable using its persistent ID).

I’ve never had to think much about all these things, because almost all my usage has been read-only, accessing existing databases (that were created through other mechanisms).

Alternative Models

I think I’m also confused about whether there’s a good way to make the object logic completely modular, separate from the storage implementation. There are several patterns to distinguish:

• What about “persistence proxy” methods, where data is not actually “unpickled” into a local object, but instead the object simply acts as a proxy that relays data requests to the persistent store… SQLRow is an example of this pattern. This doesn’t fit the standard getstate / setstate model. The persistence issue gets pushed down to the level of each individual attribute, which really means assuming that attributes are “natively” persistent, i.e. types that the transport mechanism can automatically convert like int, str, float.

• “fine-grained persistence”: Writable classes like TupleRW and wildfire Row write each attribute back to the database immediately; there is no operation to “save the complete persistent state”.

We often prefer to use these patterns over a “standard pickling” model for object-relational interfaces, for several reasons:

• memory & speed: for really large datasets the standard Python attribute model (dict) uses up too much memory. A tuple or slots can use five-fold less memory, and also be much faster.

• write behavior: if the database is “writable”, then changing an attribute should be propagated to the database (and probably right away). That implies a descriptor. On the other hand, if the database is read-only, then we again need a descriptor to raise an AttributeError on any setattr attempt.

• if we want “fine-grained” interaction between the local object representation and a persistent store (which is typically what you want for an object-relational model).

Whole Object Persistence vs. Attribute-level Persistence

This all seems to boil down to just two basic patterns:

• conventional unpickling / pickling takes the whole object as the unit of persistence: take data out from database, use it until done, then save back to database. This works OK for a read-only pattern (but doesn’t raise write errors as it should).
• attribute-level persistence: the persistence problem is pushed down to the individual attributes, typically with a descriptor handling all the requests. Any kind of proxy or fine-grained persistence uses this pattern.

Descriptor Categories

There are several kinds of descriptor usages that come up again and again:

• proxy: forwards query to someone else, typically a remote server

• computed: computes the requested attribute either by external calls or based on other attributes of the object

• read-only: raises appropriate exception if user tries to write the attribute. May also provide attribute interface to a more efficient internal storage mechanism (such as slots or tuple)

• write-consequences: writing to this attribute triggers an action, such as value checking, saving to an external server, or other “consequences”

Feynman achieves a remarkable combination of intellectual engagement — plugging you into the fundamental ideas of a problem area — and accessibility. Conventional textbook treatment uses formalism and jargon to elevate the author and distance the reader from the material. It feels like you are being inducted into the holy mysteries… which puts most people to sleep. Instead of stimulating your own questioning of the material, it implies that such Difficult and Important Ideas will require long, hard hours of study to get even a glimmer. Feynman could easily come across as “too smart for a normal human to understand”, but unlike some writers, that’s not what he wants. Somehow he is able to prick that bubble effortlessly and give you the feeling of a wonderful tour guide who is going to show you his favorite marvels. No barriers of jargon or “obfuscation-sophistication” get in the way of understanding him. He simply refuses the conventional academic tone. He doesn’t believe in it!

There is much for me to learn from Feynman’s example.

• One key to Feynman’s accessibility is his ability to break a complex body of material into short, self-contained “lectures” (especially in the 3 volume Feynman Lectures in Physics). Each lecture feels like a little guided tour, with very little barrier to entry, rather than “yet another chapter in a hefty textbook”. He takes your hand and whisks you off on the tour. Each chapter is only 8 - 10 pages long, which would probably be about 6 - 8 of my laTex pages, given the generous margins reserved in Feynman for his figures.

• He keeps asking questions so basic that even a freshman would be embarrassed to ask them (for fear of being considered a simpleton). By asking fundamental questions, Feynman both reveals how interesting and useful they are, and liberates you (the reader) to ask such questions yourself.

• This plays an important role in making each chapter approachable and self-contained. Each chapter starts with a thought-provoking introduction that asks basic questions anyone can understand — even if they haven’t read the preceding chapters. Even though the rest of the chapter gradually gets more technical (and requires background from previous chapters), he continuously re-raises the fundamental questions. Thus, it becomes possible to read Feynman on several different levels. You could read it just for the fundamental questions, observations and theories he presents so intuitively. You could actually learn a lot about physics even if you made little effort to master the equations. Since his math is as clear (and presented as simply) as his text, it will give you another level of understanding. At any rate, a principle emerges: fundamental questions make the material more approachable and easier to understand (on multiple levels), both by exciting our interest and by giving us the “bigger picture” in which we can see how each detail fits. Technicalities become integrated rather than fragmented.

• Engaging the reader with fundamental questions can shift his/her relationship with the material from passive to active. Typical undergraduate coursework emphasizes “knowing the right answer”. Feynman shows you the importance of asking the right question, and implies that you too could do this, with a little common sense and imagination. Most readers won’t make that leap, but some will be inspired.
• His tone is conversational. He uses exclamation points and italics liberally, turning the textbook into a conversation with an enthusiast. He peppers each chapter with dialogs that highlight possible confusions and subtleties. For example in a chapter selected at random (8), we find the following dialogs over the course of a few pages:
  • “Now, you may say, ‘This is all some kind of trivia,’ and indeed it is. How can we describe such a one-dimensional motion…?”, which leads into a step by step description of a car’s motion.
  • “Perhaps you say, ‘That’s a terrible thing–I learned that in science we have to define everything precisely.’ We cannot define anything precisely! If we attempt to, we get… philosophers… one saying… ‘You don’t know what you are talking about!’ The second one says, ‘What do you mean by know? What do you mean by talking? What do you mean by you?,’ and so on.”
  • “Zeno produced a large number of paradoxes… ‘Listen,’ he says, ‘to the following argument: Achilles runs 10 times as fast as a tortoise…’”
  • “In order to get to the subtleties in a clearer fashion, we remind you of a joke… the cop comes up to her and says, ‘Lady, you were going 60 miles an hour!’ She says…” He follows out the implications of this dialog for 3 paragraphs.
  • The rest of the chapter follows this spirit of question and answer. He asks a question, tries out an answer, asks another question… The only difference is that this dialog is not quoted as a conversation between two named characters. Instead, this dialog is between Feynman and the reader. He always addresses the reader as “you”, as if he were writing a letter rather than a textbook. He always speaks as “we” (”Now we have to discuss the inverse problem… Let us say, ‘In the first second her speed was such and such…’ ” )

• Feynman must have made a conscious decision to favor concepts over technicalities, and to govern the latter with a strict rule of necessity and clarity. His use of math is lucid and compact; he never allows it to take over the text or obscure the fundamental questions. His equations illuminate, and with the minimum effort necessary. This seems only sensible for a freshman course!
• Unlike many theorists, Feynman highlights the difference between science and math, and unabashedly declares his purpose to be scientific. His fundamental questions are about the universe and how it works, not about equations and what we can prove about them. In his lectures math comes second, only when needed to clarify a physics problem. Many theorists don’t see it this way. They’re mathematicians first and foremost, devoted mainly to their mathematical model and techniques, and view the physics (especially its empirical side) as a bastardized approximation of the mathematical beauty that interests them. The machismo (and ego) of mathematical rigor often drives this tendency into places where it doesn’t really help the student. A physics class can easily become a watered-down math course; all you have to do is bring in your favorite mathematical methods, and before you know it… Feynman vigorously refuses this path.

I’ve also posted some background material cut from different chapters of my draft textbook as a PDF.

• Chapter 1 draft introduction to probabilistic inference.
• Paper on identification of SNPs from EST chromatogram data.

I think Silverlight potentially has a lot of importance for Python for several reasons. Its basic function is fairly amazing: it lets you run Python inside your web browser with complete control over the web page contents and interactivity. Those of us who feel that the web browser is the logical place for many kinds of interactive documents, visualization and rapid prototyping, but want to use a real language (instead of Javascript!), finally have a solution. This is exciting! It is thrilling to be creating interactive web pages whose interactivity is all done by Python code — forget all that JavaScript!

But Pythonistas should also pay attention to Silverlight as a potential threat: this is Microsoft’s attempt to hijack Python’s many virtues and put them to work for Microsoft. For example, Silverlight runs IronPython, and as such lacks most of the Python standard libraries that Pythonistas know and rely on. In their place it gives you access to .NET. Similarly, it redirects you away from open standards like SVG and into Microsoft’s WPF (Windows Presentation Framework?). The concern here is that huge numbers of .Net and web developers will be taught a Python that is Not Python, i.e. they will be taught to write code that can’t run in standard Python, or indeed anywhere outside of Microsoft’s IronPython. The good news, I think, is that Python is too dynamic to box in this way; it is very easy to port Python standard libraries to run in Silverlight. I think Pythonistas should push very hard to provide as complete as possible a standard library for IronPython and Silverlight, so that developers in those environments are not forced over to the Dark Side.

Another interesting possibility is that Silverlight is the “restricted execution environment” Python has always lacked. Python code running in Silverlight supposedly has no access to the file system or system calls, and no way to read or alter user files. Since it runs in the web browser, and executes code sent by potentially malicious websites, logically it would have to be secure in this sense, or it will be worse than useless for Microsoft’s purposes. Taking a cynical view, if Silverlight takes off, hackers are going to be attacking it from every angle, and Microsoft will be forced to patch up the holes. So over time, Silverlight could evolve into a safe “sandbox” where you can run code that you want to confine to very limited privileges.

OK, so what have I done with Silverlight?

• I ported a fair number of missing standard library modules to get my own code working. The most significant are pickle, types, xmllib, xmlrpclib, and urllib (only some functions; Silverlight already provides http connection objects). This was amazingly easy.
• I mainly did this to create an XML-RPC connection back to the server. Silverlight 1.1 alpha refresh is restricted to only access the same server that the code was retrieved from. I’ve written a combined HTTP/XML-RPC server (in Python, naturally, it’s dead easy) so that my Python code in Silverlight can communicate with the server via XML-RPC. This is incredibly easy, just as you’re used to doing in Python:
  import xmlrpclib
server = xmlrpclib.ServerProxy('http://localhost:8000')
result = server.some_method(*args)
  I will make all this available so Pythonistas can start playing with this by quickly launching the server and simply pointing their web browser at it.
• I’ve got almost all of Pygr, including Pygr.Data, running in Silverlight. The one module that required some porting was cnestedlist, which was written in a combination of C and Python. I had to convert some things to pure Python and get rid of other things that aren’t relevant in Silverlight (e.g. file IO).
• I have pygr.Data working great in Silverlight. That is, the server is running an XML-RPC pygr.Data server, and in Silverlight I can just import pygr.Data as usual and immediately do everything I would usually do i.e. get both sequence databases / sequences and alignment databases (NLMSA) and work with them exactly as if I were in my usual Python interpreter. This is a little mind blowing. I can’t wait to demo this with access to a 28 vertebrate genome alignment in Firefox!
• I’ve been learning all this in part via Michael Foord’s very helpful Silverlight / IronPython pages. I used his simple interactive console code to try all this out by hand. But I hit a lot of crashes that killed Firefox. At first I just thought Silverlight 1.1 alpha is really buggy, but finally I realized that the problem is not Silverlight itself, but apparently the interactive console. When I just run my code directly in Silverlight (not through his interactive console) the crashing goes away. I wrote my own much simpler interactive console to eliminate all that crashing.
• Things I haven’t figured out: proper traceback printing; Silverlight’s odd handling of hierarchical module names. It doesn’t seem to be able to handle them, so to start with I just put the pygr modules in a single directory and access them as import seqdb instead of from pygr import seqdb. I had to kluge pickle.unpickling to follow this convention too.

I will post a tar of the source code tomorrow, and update this page to link to it. But now to bed.