unofficial planet python

S.Lott: Voice of the Customer

Sun, 06 Jul 2008 19:06:21 +0000

Or What do people want in a book on programming?

Voidspace: Diagramming on the Mac

Sun, 06 Jul 2008 16:25:21 +0000

One of the annoying things about writing a book is having to create my own diagrams. This was something I wasn't expecting when I started the project, I'm a good writer but awful at producing diagrams. ... [530 words]

Chuck Thier: Things you didn't know about ninjas

noreply@blogger.com (Chuck) — Sun, 06 Jul 2008 05:25:17 +0000

In trying to learn more about how to process the human language and how it can be applied in the context of a game, I recently ran across an interesting knowledge base called ConceptNet. ConceptNet is a freely available commonsense knowledgebase and natural-language-processing toolkit which supports many practical textual-reasoning tasks over real-world documents right out-of-the-box It comes

James Tauber: On Intelligence and the HTM Workshop

Sun, 06 Jul 2008 04:43:05 +0000

Like a lot of geeks, I've been interested in how the brain works for most of my life. Artificial Intelligence was always one of my interests within computing (and part of what got me interested in linguistics at a very early age).

Within my linguistics research, I've always been interested in models that are biologically plausible so it was a huge delight to read Jeff Hawkins' On Intelligence back in early 2005 and find a theory that was biologically-based and believable from a linguistics point of view. One prominent psycholinguist told me in 2006 that it was one of the most promising theories he'd ever read.

After reading the book, I promptly went out and built a library (as I am wont to do) of about 20 books on general neuroscience, computational neuroscience and the relationship between the brain and language. I started thinking about how to implement the ideas and, after reading some of Jeff's and Dileep George's early papers, augmented the library further with books on Bayesian networks, belief propagation, etc.

When Jeff and Dileep started Numenta and eventually released an early version of their Hierarchical Temporal Memory (HTM) platform in Python, I was particularly excited to try it out, in particular applying it to linguistics. I started the htm-ling mailing list to gather other people interested in applying HTM to models of language. It turned out to be hard to get word out to other people interested in HTM and linguistics, however.

I never got very far with Numenta's code, mostly because there were just too many other things I was working on.

But then a couple of months ago, I found out Numenta was running a workshop / conference. I thought it would be an excellent opportunity for me to (a) get back up to speed with what Numenta was doing and how to use their NuPIC platform; (b) meet other people interested in applying HTM to linguistics.

So a couple of weeks ago, I attended the first Numenta HTM Workshop. I had a great time. It was great to meet Jeff and the rest of the team. Dileep's talk on the algorithms in NuPIC was particularly helpful to me in understanding how things work.

There were a number of people who expressed an interest in the application to linguistics so in the evening I ran a BOF. None of the attendees (as far as I could tell) were linguists by training so I didn't really get to talk too technically from a linguistics perspective. The boost to the mailing list membership hasn't created any more discussion yet either.

But I am still hopeful that an HTM-like approach (whether in the form of NuPIC or some other implementation) might be useful in building biologically-plausible models of language processing.

Ali Afshar: The problem with jQuery

noreply@blogger.com (Ali) — Sun, 06 Jul 2008 01:04:13 +0000

Ok, I'll start by saying that I love jQuery as much as the next developer, and every contributor has my eternal thanks, but there is a problem.

Plugin development goes crazy, in that many similar plugins are released, and often these are just modifications including other sets of modifications. There must be a better way, like isn't this what version control is made for?

Here is an excerpt of http://docs.jquery.com/Plugins#Forms

* Autocomplete by Dylan Verheul
Autocomplete with caching to limit server requests and other options.

* jQuery Autocomplete Mod by Dan G. Switzer
Modification of Autocomplete plugin with enhancements and bug fixes.

* Modified Auto-complete by Anjesh Tuladhar
Auto-complete plugin extended for auto-completing multiple words in the same text input

* Autocomplete by Jörn Zaefferer
Heavily improved Dylan Verheul’s initial plugin, integrating modifications by Dan G. Switzer and Anjesh Tuladhar.

* jQuery Autocomplete by Saurabh Periwal
Modification of Autocomplete plugin with enhancements of onItemSelect and Multiselect.

* Jeditable + Autocomplete by Ritesh Agrawal
Extended jeditable to include option for autocomplete. Also extended Dylan Verheul's autocomplete javascript to include option for having input token separator. Check the demo.

End of snippet.

Duncan McGreggor: Native LoadBalancing for Twisted Apps

noreply@blogger.com (Duncan McGreggor) — Sat, 05 Jul 2008 21:51:40 +0000

Yesterday, right before midnight, I tagged the 1.1.0 release of txLoadBalancer on Launchpad after completing the last of the planned features. There are some pretty radical changes that have been developed for this release... and the coolest part is this is just the beginning :-) (See the TODO if you don't believe me!)

You can checkout from lp:~oubiwann/txloadbalancer/1.1.0 or download from PyPI. If you're a PyPI expert, I've got some questions for you at the end of this post... Been having some sucky experiences with PyPI lately :-(

So here's what's going on with txLoadBalancer:

Improved API

The biggest thing you'll notice if you've switching from PythonDirector is the massive overhaul the API has undergone. Things are cleaner and generally more modern, with a concise and well-defined module layout.

New Load Balancing Algorithm

I've added support for a weighted host scheduler. Given a weight that represents the frequency a host should be used, a host will be randomly selected, based on it's weight. For example, with two hosts, one having a weight of 1 and the other having a weight of 3, host 2 will be chosen about 75% of the time and host 1 will get about 25% of the requests.

Right now, this algorithm has to make several calls to other parts of the code in order to get all the data it needs (it also builds some crazy iterators). As such, it's rather slow and performs poorly when compared to the very light-weight least-connections algorithm. That being said, the next release will include optimizations for the weighted scheduler that make use of a Twisted timer and caching.

Native Twisted Load-Balancing

Here's the sexiest part: you can now load-balance your Twisted application by using the txLB API; you don't even need to run the load-balancer as a separate app! This evolved as a feature after a conversation with an as-yet unnamed cloud hosting provider, a follow-up discussion with the Divmod team, and then some quiet pondering about ways in which Twisted applications could be supported in cloud/grid/massively-multi-core architectures.

The "self load-balancing" API in txLB is not a comlete solution for grid-hosting, but it is a first step in one direction (we've been discussing lots of others, too, including the use of our deployment tool).

Before I show you how to use the self load-balancing API, let's take a quick look at a normal Twisted application service:

from twisted.web import static, server
from twisted.application import service, internet

application = service.Application("Demo Web Server")
web = server.Site('/home/oubiwann/public_html')
service = internet.TCPServer(7001, web)
service.setServiceParent(application)

You start that with the command twistd -noy myweb.tac. For use with the next example, you can also start two more, one on port 7002 and the other on port 7003.

Now here's what you do to make a self load-balanced app:

from twisted.application import service

from txlb import manager
from txlb.model import HostMapper
from txlb.schedulers import leastc
from txlb.application.service import LoadBalancedService

proxyServices = [
  HostMapper(proxy='127.0.0.1:8080', lbType=leastc, host='host1',
      address='127.0.0.1:7001'),
  HostMapper(proxy='127.0.0.1:8080', lbType=leastc, host='host2',
      address='127.0.0.1:7002'),
  HostMapper(proxy='127.0.0.1:8080', lbType=leastc, host='host3',
      address='127.0.0.1:7003'),
]

application = service.Application('Demo LB Service')
pm = manager.proxyManagerFactory(proxyServices)
lbs = LoadBalancedService(pm)
lbs.setServiceParent(application)

As you would expect, you need to indicate the proxy host:port, the algorithm to use, and the hosts that are to be balanced. The host setup assumes that you have three services running on localhost ports 7001, 7002, and 7003. All that's needed now is to just run that code with the usual twistd -noy myapp.tac. Also, for demonstration purposes, this is a somewhat simplified example of what is possible.

This may seem like a lot of extra work when compared to the simple web host above, but think about it: we're load-balancing here :-) This saves you from having to manage yet another application. With a few extra lines of code, you can keep it all in one place and have it manage itself.

Note that this API is in development and continuing to improve. The example above is from code running in trunk. For the more verbose configuration that is in the 1.1.0 release, be sure to see ./bin/txlbWeb.tac from the source tarball. To play with the latest and greatest, you'll want to checkout the code here: lp:txloadbalancer.

Other Goodies

Here is some other good stuff in the release:

You can now ssh into a txLB instance and mainipulate the load-balancer in real time from an interactive Python interpreter.
You can change the proxy to listen on a different port while the application is running (no restart requred!).
Changes made to the configuration while running are no longer volatile; they are saved to disk (and your old config gets backed up).
Work from Apple, Inc. was included in this release, too (they use the old PythonDirector in their Calendaring server). This includes a bug fix and management socket feature.
There is a significant jump in performance between this release and the previous one. I believe this to be due to the separation of concerns in the API, but haven't yet confirmed that.

Coming Work

There are a lot of exciting features coming for txLB. Just to name a few:

improved weighted algorithm
resources-based algorithm (a scheduler that determins the weight of a proxied host by memory, CPU, etc., utilization)
smarter proxied host failover and recovery
a heartbeat manager
txLB-powered application cloning (when started, an app will determine if it needs to run the clone as the managing load-balancer or simply as a proxied host)
auto-discovery of balanced hosts
proxy fail-over (a balanced host taking over as manager in the event that the manager goes down)
ApacheMQ/Stomp integration
LDAP/RADIUS authentication

Additionally, I'll be putting together some basic performance metrics contrasting Apache and load-balanced Twisted apps. I will also be comparing previous versions of txLB/PythonDirector with the latest release(s).

Problems with PyPI

I will close this post on a sad note: PyPI used to be an amazing experience for me (a couple years ago, when it was still being called "cheeseshop"). Everything worked as it was supposed to. This hasn't been the case when I've used it recently (over the past few months).

For all that I say about PyPI, I allow for the fact that I may just be missing something, and it may be entirely my fault. That being said, I spent about 3 hours online last night combing though the SIG mail list, the bug list on sourceforge, and blog posts about setuptools and PyPI, and could find no answers to my questions. Well, with the possible exception of a bug report, but it doesn't look like it was confirmed by a PyPI team member, so I'm not sure if it's valid or not.

Here are my issues:

When I upload my project using python setup.py [sdist|bdist_egg] upload, no metadata defined in my setup() function is presented on my package's PyPI page. When I click the metadata link, it's only got three sparse lines.
When I manually upload from the package's PKG-INFO itself, all the metadata is presented on the page as it should be, with the exception of the long description. It is in plain text instead of ReST (I am checking that it is valid ReST using distutils settings of reporter.halt_level = 5, reporter.report_level = 1, settings.pep_references = False, and settings.trim_footnote_reference_space = None; these are the same settings that Zope Corp uses to verify the ReST that it uploads to PyPI).
When I manually edit the long description in the form, I get the same thing: plain text, no ReST.
When I upload a package that is displayed properly on PyPI (such as zc.twist; uploaded as one of my projects by chaning the name), I get the same problem (this is why I think it might be something that I'm doing wrong...): no metadata, and when I upload the PKG-INFO manually, no ReST.

Why, oh why, cruel fates, does this not work any more? I used to be able to upload to PyPI without any of these issues...

Simon Willison's Weblog: OSM routing, A*, cycle-filtered, python

Sat, 05 Jul 2008 15:13:48 +0000

OSM routing, A*, cycle-filtered, python (via). A python library for finding routes using OpenStreetMap data.

Armin Ronacher: virtualenv to the Rescue

Sat, 05 Jul 2008 14:16:33 +0000

Deploying Python applications is not that hard, but there are some pitfalls. The main problem everyone stumbles upon sooner or later is that sys.modules is a singleton. Modules are cached there which the effect that there can be exaclty one version of a library loaded at the same time per interpreter. There is no way around that and it’s fine. However there is another problem: Everyone installs everything into a system wide singleton called site-packages. Sooner or later everything ends up there, and while pkg_sources supports switching between versions this is very unconfortable and requires extra hackery.

My plan to solve these problems always was creating a folder, putting all libraries into that folder and site.addsitedir()ing that folder in the file that starts up the application. But especially for web applications that’s a dull repetitive task and easy_install and other libraries don’t play well with this approach.

Fortunately Ian Bicking wrote a really cool module called “virtualenv” that creates virtual Python environments. And it does that in a very cool manner. The following example shows how we run LodgeIt on our server:

First you need to install virtualenv. This assumes you already have easy_install:

mitsuhiko@nausicaa:~$ sudo easy_install virtualenv
Searching for virtualenv
Reading http://pypi.python.org/simple/virtualenv/
Best match: virtualenv 1.1
Downloading virtualenv-1.1-py2.5.egg
Processing virtualenv-1.1-py2.5.egg
creating /opt/local/lib/python2.5/site-packages/virtualenv-1.1-py2.5.egg
Extracting virtualenv-1.1-py2.5.egg to /opt/local/lib/python2.5/site-packages
Adding virtualenv 1.1 to easy-install.pth file
Installing virtualenv script to /opt/local/bin

Installed /opt/local/lib/python2.5/site-packages/virtualenv-1.1-py2.5.egg
Processing dependencies for virtualenv
Finished processing dependencies for virtualenv

Now we can create a folder folder the application. In our case the folder is called after the domain where the web application runs:

mitsuhiko@nausicaa:~$ virtualenv paste.pocoo.org
New python executable in paste.pocoo.org/bin/python
Installing setuptools.......................done.

The virtual python is now available in that folder and we can switch into that environment by sourcing the activate file:

mitsuhiko@nausicaa:$ cd paste.pocoo.org/
mitsuhiko@nausicaa:~/paste.pocoo.org$ . bin/activate
(paste.pocoo.org)mitsuhiko@nausicaa:~/paste.pocoo.org$

From that point onwards “python” and “easy_install” point to the executables in the ~/paste.pocoo.org/bin folder. The prompt is prefixed with the name of the virtual environment so that we are reminded that “python” is our virtual Python. We can now install all the dependencies:

$ easy_install SQLAlchemy==0.4.4 Jinja2 Werkzeug Pygments

Half a minute later the virtual python interpreter will now have the libraries available and we’re ready to add out application:

$ hg clone http://dev.pocoo.org/hg/lodgeit-main

Lodgeit itself doesn’t have a setup.py file but we can just cd into the folder and run the testing server to test if it works:

$ cd lodgeit-main/
$ python runlodgeit.py runserver
 * Running on http://localhost:5000/
 * Restarting with reloader...

That works perfectly for scripts invoking “python” directly such as standalone servers like CherryPy but won’t work for mod_wsgi where the interpreter is created by mod_wsgi and doesn’t point to the virtual environment.

There are two ways to solve the problem. For mod_wsgi 2.0 you can run the web application in daemon mode and pass it the path to the site packages in the apache config:

WSGIDaemonProcess lodgeit python-path=/path/to/site-packages

Alternatively you can go into the .wsgi file and add the site-packages by hand:

import site
site.addsitedir("/path/to/site-packages")

The site-packages of the virtual environment are located at /path/to/virtual/env/lib/python2.X/site-packages.

Even though I was not interested in virtualenv in the beginning I changed my mind because it’s straightforward to use and easy to maintain. More information about virtualenv is available in the Cheeseshop and the announcement blog post in Ian’s blog.

Andrew Dalke: Testing the bitslice algorithm for popcount

Sat, 05 Jul 2008 12:00:00 +0000

This is part 7 of an on-going series dealing with molecular fingerprints:

Molecular fingerprints, background
Generating molecular fingerprints with OpenBabel
Computing Tanimoto scores, quickly
Block Tanimoto calculations
Faster block Tanimoto calculations
HAKMEM 169 and other popcount implementations
Testing the bitslice algorithm for popcount

Comments?

I want to compute the popcount of molecular fingerprints containing 1024 bits. My algorithm has been to compute the popcount 8, 16, 32, or 64 bits at a time and repeat that for all of the words in the fingerprint. There are a couple of algorithms which compute the popcount of multiple words in parallel by doing a partial popcount of each word and combining the intermediate results to get the result over all words.

Lauradoux Cédric wrote a nice overview on Hamming weight, including explanations of the tree merging and bit-slicing algorithms. In personal email he pointed out this page from comp.arch.arithmetic where someone attributes the algorithm to Robert Harley.

It took me a while to understand it. It's been rather a while since I had to do much thinking about half-adders. About 17 years. But it's not hard. I'm not going to explain it though. It's best in pictures and that will take me too long to do. You can follow those links though.

While he hasn't finished the overview explanation on how to combine the two algorithms, his Hamming weight page includes source.

I've added two bit-slicing implementations to my popcnt.c benchmark. The algorithm from comp.arch.arithmetic processes 7 words at a time and the one from Cédric processes 24 words at a time. Both fall back to other methods for the last few words if the input isn't an exact multiple of 24 or 7.

Cédric implemented several algorithms in his code distributions and included programs to compare their performance. The results suggested that the 24 word bitslice algorithm would be twice as fast as as the 16 bit looup table. I implemented it and found that while it was fast, and in my laptop even faster than the 16 bit lookup table, it was only about 10% faster and not a factor of 2.

% sysctl machdep.cpu.brand_string
machdep.cpu.brand_string: Intel(R) Core(TM)2 CPU         T7600  @ 2.33GHz
% gcc --version
i686-apple-darwin8-gcc-4.0.1 (GCC) 4.0.1 (Apple Computer, Inc. build 5367)
   ... deleted ...
% cc -O3 popcnt.c
% ./a.out
FreeBSD version 1   :  3854904 us; cnt=32511665
FreeBSD version 2   :  3517953 us; cnt=32511665
16-bit LUT          :  2127718 us; cnt=32511665   was the fastest
8-bit LUT           :  5317244 us; cnt=32511665
8-bit LUT v2        :  3762467 us; cnt=32511665
Wikipedia #2        :  5493756 us; cnt=32511665
Wikipedia #3        :  5295541 us; cnt=32511665
gcc popcount        :  6242596 us; cnt=32511665
gcc popcountll      : 10402364 us; cnt=32511665
HAKMEM 169/X11      :  4305536 us; cnt=32511665
Bitslice(7)         :  2077473 us; cnt=32511665
Bitslice(24)        :  1843217 us; cnt=32511665  fastest
naive               : 23460323 us; cnt=32511665
Wegner/Kernigan     : 14976014 us; cnt=32511665
Anderson            : 63971600 us; cnt=32511665
8x shift and add    : 21728414 us; cnt=32511665
32x shift and add   : 19747808 us; cnt=32511665

I dug around for a while and used Toby's comment to try Shark as a profiler. Things made a bit more sense after I remembered to add the '-g' option to gcc so I could get Shark to line up the code and assembly side-by-side. My conclusion is that Cédric's code has too much function call overhead because the 16-bit LUT is being called for every 32 bit number, while the 24 word bitslice code is only called every 24 words. My code doesn't have that overhead, which is why the times are closer to each other.

As before, I wondered how much "-m64" affects things. I'll declare it a tie between the new bitslice code and the old 16-bit lookup table. (While though the numbers aren't the same, the difference is about the same as when I run the same algorithm again.)

% cc -O3 popcnt.c -m64
% ./a.out
FreeBSD version 1   :  3902385 us; cnt=32511665
FreeBSD version 2   :  3422655 us; cnt=32511665
16-bit LUT          :  1621557 us; cnt=32511665 fastest
8-bit LUT           :  2070320 us; cnt=32511665
8-bit LUT v2        :  2094362 us; cnt=32511665
Wikipedia #2        :  2135130 us; cnt=32511665
Wikipedia #3        :  2142576 us; cnt=32511665
gcc popcount        :  8128024 us; cnt=32511665
gcc popcountll      :  4087390 us; cnt=32511665
HAKMEM 169/X11      :  5805272 us; cnt=32511665
Bitslice(7)         :  1728262 us; cnt=32511665
Bitslice(24)        :  1698257 us; cnt=32511665 fastest
naive               : 24990997 us; cnt=32511665
Wegner/Kernigan     : 16864887 us; cnt=32511665
Anderson            : 12463648 us; cnt=32511665
8x shift and add    : 21715482 us; cnt=32511665
32x shift and add   : 20003160 us; cnt=32511665

All of this was on one processor and compiler. I decided to try another machine with a slightly different processor and an earlier version of gcc.

%sysctl -w hw.model
hw.model: Intel(R) Core(TM)2 CPU          6420  @ 2.13GHz
%gcc --version
gcc (GCC) 3.4.6 [FreeBSD] 20060305
    ...
%cc popcnt.c -O3 
%./a.out 
FreeBSD version 1   :  4270677 us; cnt=32511665
FreeBSD version 2   :  3785047 us; cnt=32511665
16-bit LUT          :  2034836 us; cnt=32511665
8-bit LUT           :  1935156 us; cnt=32511665   fastest
8-bit LUT v2        :  1987370 us; cnt=32511665
Wikipedia #2        :  5010639 us; cnt=32511665
Wikipedia #3        :  4931266 us; cnt=32511665
gcc popcount        :  5793299 us; cnt=32511665
gcc popcountll      :  6815575 us; cnt=32511665
HAKMEM 169/X11      : 10788366 us; cnt=32511665
Bitslice(7)         :  2467150 us; cnt=32511665
Bitslice(24)        :  2469399 us; cnt=32511665
naive               : 27722786 us; cnt=32511665
  ... too slow to worry about ...

In this case I would use the 8-bit lookup table, which is about 20% faster than the the bitslice code.

Here's the results on an older Pentium 4 (Prescott) CPU 2.80GHz with gcc 3.4.6:

FreeBSD version 1   :  3745509 us; cnt=32500610
FreeBSD version 2   :  3774150 us; cnt=32500610
16-bit LUT          :  2762583 us; cnt=32500610
8-bit LUT           :  2430362 us; cnt=32500610    fastest
8-bit LUT v2        :  2433684 us; cnt=32500610
Wikipedia #2        : 11509758 us; cnt=32500610
Wikipedia #3        :  8540921 us; cnt=32500610
gcc popcount        :  9100055 us; cnt=32500610
gcc popcountll      : 10568664 us; cnt=32500610
HAKMEM 169/X11      : 10970609 us; cnt=32500610
Bitslice(7)         :  4226871 us; cnt=32500610
Bitslice(24)        :  3700020 us; cnt=32500610
naive               : 36117662 us; cnt=32500610
Wegner/Kernigan     : 46479326 us; cnt=32500610
Anderson            : 174285325 us; cnt=32500610
8x shift and add    : 90028247 us; cnt=32500610
32x shift and add   : 86390410 us; cnt=32500610

Bitslice(24) is still the fastest bit-twiddler solution, but quite a bit slower than using an 8-bit lookup table. There is no single best solution. It seems the code is also compiler dependent so I'm installing GCC 4.2 to see how that affects things. That'll probably take a few hours so it's time to sleep.

Okay, fink installed gcc 4.2.2 (the most recent is 4.3.1 but I decided it wasn't worth the time to compile it myself). Here's the numbers on my laptop for 32-bit:

% gcc-4 -O3 popcnt.c
% ./a.out 
FreeBSD version 1   :  4161621 us; cnt=32511665
FreeBSD version 2   :  2930977 us; cnt=32511665  was 3517953
16-bit LUT          :  2534380 us; cnt=32511665  was 2127718
8-bit LUT           :  4472733 us; cnt=32511665
8-bit LUT v2        :  4499285 us; cnt=32511665  was 3762467
Wikipedia #2        :  5031855 us; cnt=32511665
Wikipedia #3        :  3874931 us; cnt=32511665  was 5295541
gcc popcount        :  4808307 us; cnt=32511665
gcc popcountll      :  5978393 us; cnt=32511665
HAKMEM 169/X11      :  4315751 us; cnt=32511665
Bitslice(7)         :  2111625 us; cnt=32511665
Bitslice(24)        :  1443337 us; cnt=32511665   fastest; was 1843217
naive               : 51557971 us; cnt=32511665
Wegner/Kernigan     : 15616245 us; cnt=32511665
Anderson            : 63424235 us; cnt=32511665
8x shift and add    : 19179445 us; cnt=32511665
32x shift and add   : 19502222 us; cnt=32511665
%

and for 64 bit

% gcc-4 -O3 popcnt.c -m64
% ./a.out
FreeBSD version 1   :  4192391 us; cnt=32511665
FreeBSD version 2   :  2812570 us; cnt=32511665    was 3422655
16-bit LUT          :  1494747 us; cnt=32511665    was 1621557
8-bit LUT           :  2182569 us; cnt=32511665
8-bit LUT v2        :  2170191 us; cnt=32511665    was 2094362
Wikipedia #2        :  2001489 us; cnt=32511665
Wikipedia #3        :  1361488 us; cnt=32511665    was 2142576
gcc popcount        :  7350367 us; cnt=32511665
gcc popcountll      :  3718116 us; cnt=32511665
HAKMEM 169/X11      :  4339839 us; cnt=32511665
Bitslice(7)         :  1693849 us; cnt=32511665
Bitslice(24)        :  1206318 us; cnt=32511665    fastest; was 1698257
naive               : 24914902 us; cnt=32511665
Wegner/Kernigan     : 15557531 us; cnt=32511665
Anderson            :  8764008 us; cnt=32511665
8x shift and add    : 21710211 us; cnt=32511665
32x shift and add   : 19583803 us; cnt=32511665
%

It looks like things are nearly always faster for 64 bit code, and sometimes faster for 32 bit code.

Michael Sparks: Concurrent software is not the problem - Intel talking about 1000s of cores

Sat, 05 Jul 2008 11:47:52 +0000

Intel have recently been talking about the fact that we'll probably have to deal with not hundreds but thousands of cores on a machine at some point in the near future (5-10 year time frame based on their 80 core prototype). Now many people seem to be worried about how to use even a dual core machine, so naturally many people are probably going wa-HUH? However I suspect the Erlang crowd have a similar reaction to us - which is "cool".

Why? Like the erlang group, in Kamaelia, the thing we've focussed on is making concurrency easy to work with, primarily by aiming for making concurrent software maintenance easier (for the average developer). In practical terms this has meant putting friendly metaphors (hopefully) on top of well established principles of message passing systems, as well as adding support for other forms of constrained shared data. (STM is a bit like version control for variables).

We've done this by using various application domains as the starting point, such as DVB, networking and use of audio/video etc, and used Python as the language of choice to do so (Though we probably could've shouted about our application uses more/better, though we've getting better I think :-). However the approaches apply to more or less any non-functional language - so there's a proof of concept of our miniaxon core in C++, Ruby, & Java as well. (C++ & ruby ones deliberately simple/naive coding style :)

This does mean that now when we approach a problem - such as the desire to build a tool that assists a child learning to read and write - we end up with a piece of code that internally exhibits high levels of concurrency. For example, even the simple Speak And Write application is made of 37 components which at present all right in the same process, but could be easily be made to use 37 processes... (prepending all Pipelines & Graphlines with the word "Process")

Despite this, we don't normally think in terms of number of components or concurrent things, largely because you don't normally think of the number of functions you use in a piece of code - we just focus on the functionality we want from the system. I'm sure once upon a time though people did, but I don't know anyone who counts the number of functions or methods they have. The diagram below for example is the high level functionality of the system:

Unlike many diagrams though, this has a 1 to 1 correspondance with the code: (skipping some details below)

bgcolour = (255,255,180)
Backplane("SPEECH").activate()

Pipeline(
    SubscribeTo("SPEECH"),
    UnixProcess("while read word; do echo $word | espeak -w foo.wav --stdin ; aplay foo.wav ; done"),
).activate()
CANVAS = Canvas( position=(0,40), size=(800,320),
                  bgcolour = bgcolour ).activate()
CHALLENGE = TextDisplayer(size = (390, 200), position = (0,40),
                           bgcolour = bgcolour, text_height=48,
                           transparent =1).activate()
TEXT = Textbox(size = (800, 100), position = (0,260), bgcolour = (255,180,255),
                text_height=48, transparent =1 ).activate()

Graphline(
    CHALLENGER = Challenger(),
    CHALLENGE_SPLITTER = TwoWaySplitter(),
    CHALLENGE_CHECKER = Challenger_Checker(),
    SPEAKER = PublishTo("SPEECH"),

    CHALLENGE = CHALLENGE,
    TEXT = TEXT,
    CANVAS = CANVAS,

    PEN     = Pen(bgcolour = bgcolour),
    STROKER = StrokeRecogniser(),
    OUTPUT = aggregator(),
    ANSWER_SPLITTER = TwoWaySplitter(),

    TEXTDISPLAY = TextDisplayer(size = (800, 100), position = (0,380),
                                 bgcolour = (180,255,255), text_height=48 ),

    linkages = {
               ("CANVAS", "eventsOut") : ("PEN", "inbox"),
               ("CHALLENGER","outbox") : ("CHALLENGE_SPLITTER", "inbox"),
               ("CHALLENGE_SPLITTER","outbox") : ("CHALLENGE", "inbox"),
               ("CHALLENGE_SPLITTER","outbox2") : ("SPEAKER", "inbox"),
               ("PEN", "outbox")        : ("CANVAS", "inbox"),
               ("PEN", "points")        : ("STROKER", "inbox"),
               ("STROKER", "outbox")    : ("OUTPUT", "inbox"),
               ("STROKER", "drawing")   : ("CANVAS", "inbox"),
               ("OUTPUT","outbox")      : ("TEXT", "inbox"),
               ("TEXT","outbox")      : ("ANSWER_SPLITTER", "inbox"),
               ("ANSWER_SPLITTER","outbox") : ("TEXTDISPLAY", "inbox"),
               ("ANSWER_SPLITTER","outbox2") : ("CHALLENGE_CHECKER", "inbox"),
               ("CHALLENGE_CHECKER","outbox") : ("SPEAKER", "inbox"),
               ("CHALLENGE_CHECKER", "challengesignal") : ("CHALLENGER", "inbox"),
    },
).run()

However, what has this got to do with 1000s of cores? After all, even a larger application (like the Whiteboard) only really exhibits a hundred or two hundred of degrees of concurrency... Now, clearly if every application you were using was written the approach of simpler, friendlier component metaphors that Kamaelia currently uses, then it's likely that you would probably start using all those CPUs. I say "approach", because I'd really like to see people taking our proofs of concept and making native versions for C++, Ruby, Perl, etc - I don't believe in the view of one language to rule them all. I'd hope it was easier to maintain and be more bug free, because that's a core aim, but the proof of the approach is in the coding really, not the talking.

However, when you get to 1000s of cores a completely different issue suddenly arises that you didn't have with concurrency at the levels of 1,5, 10, 100 cores. That of software tolerance of hardware unreliability, and that, not writing concurrent software is the REAL problem.

It's been well noted that Google currently scale their applications across 1000s of machines using Map Reduce, which fundamentally is just another metaphor for writing code in a concurrent way. However, they are also well known to work on the assumption that they will have a number of servers fail every single day. This will will fundamentally mean half way through doing something. Now with a web search, if something goes wrong, you can just redo the search, or just not aggregate the results of the search.

In a desktop application, what if the core that fails is handling audio output? Is it acceptable for the audio to just stop working? Or would you need to have some mechanism to back out from the error and retry? It was thinking about these issues early this morning that I realised that what you you need is a way of capturing what was going to be running on that core before you execute it, and then launch it. In that scenario, if the CPU fails (assuming a detection mechanism) you can then restart the component on a fresh core.

The interesting thing here is that ProcessPipeline can help us out here. The way process pipeline works is as follows. Given the following system:

ProcessPipeline( Producer(), Transformer(), Consumer() ).run()

Such as:

ProcessPipeline( SimpleFileReader(), AudioDecoder(), AudioPlayer() ).run()

Then ProcessPipeline runs in the foreground process. For each of the components listed in the pipeline, it forks, and runs the component using the pprocess library, with data passing between components via the ProcessPipeline (based on the principle of the simplest thing that could possibly work). The interesting thing about this is this: ProcessPipeline therefore has a copy of each component before it started executing. Fundamentally this allows process pipeline to be able (at some later point in time) to be able to detect erroneous death of the component (somehow :) ), either due to bugs, or hardware failure, and to be able to restart the component - masking the error from the other components in the system.

Now, quite how that would actually work in practice, I'm not really sure, ProcessPipeline is after all experimental at present, with issues in it being explored by a Google Summer of Code project aimed at a multi-process paint program (by a first year CS student...). However, it gives me warm fuzzy feelings about both our approach and it's potential longevity - since we do have a clear reasonable answer as to how to deal with that (hardware) reliability issue.

So, whilst Intel may have some "unwelcome advice", and people may be reacting thinking "how on earth do I even structure my code to work that way", but the real problem is "how do I write application code that is resilient to and works despite hardware failure".

That's a much harder question, and the only solution to both that I can see is "break your code down into restartable, non-datasharing, message passing, replaceable components". I'm sure other solutions either exist or will come along though :-) After all, Kamaelia turns out to have similarities to Hugo Simpon's MASCOT (pdf, see also wikipedia link) which is over 30 years old but barely advertised, so I'm sure that other approaches exist.

Second p0st - Phillip Pearson: MySpace: "Authentication failed. Failed to resolve application URI" solution

Sat, 05 Jul 2008 08:42:29 +0000

I'm hacking on a MySpace OpenSocial app at the moment and have wasted several hours trying to debug this: all REST requests to http://api.myspace.com/ give me the error Authentication failed. Failed to resolve application URI "my oauth key here".

In the end the solution was to create a new app; the REST call succeeded immediately with the new app ID. Presumably some database migration failed on MySpace between me creating my first app (many months ago) and now, leaving my application unusable.

Comment

Mike Fletcher: IndexedFaceSet compilation improving

nospam@example.com (Mike Fletcher) — Sat, 05 Jul 2008 05:04:05 +0000

Working on the IndexedFaceSet support this afternoon and evening (with a birthday party for a friend in the middle). IndexedFaceSet's are very generic ways of defining face geometry, which makes it easy to hand-edit VRML files with them. Unfortunately, that generality means they are also rather involved to prepare for rendering, you basically have to do a lot of up-front compilation to turn the VRML97 mesh into something OpenGL compatible.

OpenGLContext has had (disabled) code to use display lists for IFS display for a very long time. It was disabled because it was slower than the array-based geometry path. The code that got used was a very generic mechanism that used the ArrayGeometry class to render the IFS. ArrayGeometry is rendered with a straightforward call to glDrawArrays().

glDrawArrays says "draw X primitives from all active arrays starting at a given index in the arrays". It allows you to use a single call to render the arrays, but the arrays have to be in "expanded" form, that is, if you have three triangles that all share a vertex (extremely common) you will wind up with 3 copies of the vertex in the arrays. The major advantage of this approach is that the code to produce the expanded form for the general case of an IFS is fairly straightforward. That is, expanding any IFS to an ArrayGeometry can be done robustly and (fairly) simply.

Obviously, however, there is a downside to the ArrayGeometry approach. The expanded form uses a lot of memory and memory bandwidth, particularly for the (extremely common) case of geometry that shares vertices. What we'd like is a way to tell the GL to render an array of indices into the active arrays, rather than a range of indices. Enter the glDrawElements function. It works like glDrawArrays, save that you specify an array of indices to render, rather than a start and count.

Thing is, glDrawElements is a bit more involved to prepare from an IFS than glDrawArrays. You have to track more data in order to map the final tessellated vertices to the values in the original, and doing the mapping in the general case doesn't make sense, as in a lot of cases you'll wind up breaking down to a fully expanded set anyway.

Luckily, we already have the fallback code . I wrote a quick test function that determines which algorithm is appropriate for a given piece of geometry and uses glDrawElements (IndexedPolygons) only when it might provide an advantage over glDrawArrays (and is easy to generate).

I still need to alter IndexedPolygons to use VBOs and push/pop attribute operations, but once that's done common modeller-created IFS's in OpenGLContext should be a little bit faster and a lot more space-efficient. At some point we could even try creating an glInterleavedArrays implementation, as the code is now set up to handle different IFS compilers.

Anyway, the code is still very rough, but I'm rather happy with the day's work.

James Tauber: Changes to Google Maps Satellite Images

Fri, 04 Jul 2008 22:51:17 +0000

It used to be obvious in Google Maps where the boundaries of different satellite images were. Each image had different brightness, contrast, colour, etc which gave away the stitching.

I always wondered whether there were techniques to normalize that.

I guess there are: Today I noticed the satellite images are stitched together seamlessly.

I also noticed some level-of-detail differences between land and ocean and that is also done pretty seamlessly.

It actually makes navigating around the satellite view a little eerie.

Anyone know when the change was made?

UPDATE: Actually it depends on the zoom level. Compare this to this. And notice the image credits are different. Interestingly, my home town of Perth looks fully normalized at all scales, even though the image sources are still TerraMetrics for the large scale and DigitalGlobe/GeoEye for the small scale.

Mike Fletcher: Smooth Movement for OpenGLContext

nospam@example.com (Mike Fletcher) — Fri, 04 Jul 2008 20:13:40 +0000

Finally got around to implementing smooth movement for OpenGLContext. Instead of just "jumping" a given distance on each press of the key, the default movement manager will now smoothly interpolate to the position (same for orientations). Code is pretty straightforward, the only tricky thing was getting holding down the key to make the motion faster.

Under the covers I'm using a pair of timers to handle the interpolation. Each time a key-press event is received the timer is restarted to interpolate between the current position and the target position. Problem was that the timers would start a frame behind, and by the time they were firing the next key-press event would be firing as well.

Result was that the cycle-fractions produced for the first frame of the animation would be about 1/2 of the time that was intended (since the key-press processing occurs between frames). Since a new keyboard event would come in that frame too, the timer would get reset all over again. Result was that pressing and holding the key would slow down the motion by about 1/2 compared with pressing discretely.

To get around that, I compress the natural fraction from the timer by 1/2 and shift it 1/2 forward. Result is that the viewer jumps forward 1/2 of the distance, then slowly adjusts into the final position if they press the key once. If they kept the key pressed, however, they tend to move forward at about 1/2 of the step distance per frame. Only real downside is that speed is once again affected by frame-rate, rather than being entirely simulation-driven. Still, it looks reasonably good and feels natural.

Titus Brown: zounds, for running lots of BLASTs

Fri, 04 Jul 2008 19:32:47 +0000

I finally got sick of manually schlepping BLAST files around, so I wrote something to do it for me. 'zounds' is a very simple server/client system for coordinating a bunch of 'worker' nodes through a central server; it does everything in Python with objects and pickling, so it's easy to do extra Python-based processing on the worker nodes. See 'filters' for more info.

You can read a bit more about zounds here:

http://iorich.caltech.edu/~t/zounds/README.html

It's freely available, open-source, etc. etc.

Comments and thoughts welcome; send them to the bip list.

--titus

Simon Wittber: First steps to GameJam!

noreply@blogger.com (Simon Wittber) — Fri, 04 Jul 2008 14:50:40 +0000

Registration is up and running at http://gamejam.org/.

There is more to the web app to be developed, but I'll hold off until enough people register their interest.

Simon Willison's Weblog: Running C and Python Code on The Web

Fri, 04 Jul 2008 08:26:27 +0000

Running C and Python Code on The Web. Adobe are working on a toolchain to compile C code to target the Tamarin VM in Flash. This will allow existing C code (from CPython to Quake) to execute in a safe sandbox in the browser.

Michael Sparks: Interesting post on requirements for project websites

Fri, 04 Jul 2008 08:06:17 +0000

I quite like this list of requirements for project websitesby Brian Jones. We've been planning a rebuild of the Kamaelia website to focus on applications written using kamaelia, how to modify them, make your apps using it, and how to join in (hoping people are interested in doing so), and whilst that's a wider scope than some of the things he's suggesting, but it's a good checklist. (After all, each of the applications themselves should actually have a project page with that sort of information).

Go Deh: Essence of Duck

noreply@blogger.com (Paddy3118) — Fri, 04 Jul 2008 05:22:52 +0000

A distillation of duck typing

"Substitution of an object with regard only of the function and signature of its run-time methods "

Lets break it down:

Substitution: Its about the later use of a new object type instead of another .
object: Its about the use of data types.
regard: What needs to be in place for things to work.
function: what the substituting object has to do to fit in.
signature: the substituting objects method signature needed for compatability.
run-time: It is what actually happens at run time that matters without the constraint of what can be determened at compile time.

There is purposefully no mention of type-checking, class hierarchy checking, in fact any 'look before you leap' checks. Note too, that I purposefully put function before signature - too many comments on duck typing focus on issues of signature and assume programmer checks for correct substitutable function are not part of duck typing.

- Paddy.

Brian Jones: This is how I want all project web sites to look…

Fri, 04 Jul 2008 00:42:47 +0000

My brain has a set of rules that software project websites get tested against. Each time a project site fails to comply with a rule, I get ever-so-slightly more annoyed, and ever-so-slightly less likely to use the software in question (if there are alternatives, this is even maybe not so “slightly”).

I thought I’d list these rules because I suspect others are like me: we’re extremely busy, we work too many hours, and are involved with too many projects to spend hours trying to figure out what some piece of code someone mentioned once in IRC actually does.

But first, know that this site actually complies with just about every single rule there is, so it’s a great template to work from if your site needs brushing up.

First and foremost, tell me, right away, what this thing does, the problem it solves, and (at a high level) the approach taken to solve the problem.
Tell me the language it’s written in. If it’s open source, and it’s written in a language I hack in, *and* it solves a problem I need solved, maybe I can help out, or be encouraged that if something flakes, I can fix it, or at least speak the developer’s language if I have to describe the issue to the folks upstream.
Tell me what OS is required, and preferably what OS/version is tested with.
Give me a full list of dependencies with links to go get them, or give me a link to “Dependencies”, or to an install document that lists them.
Tell me the current version, and the date it was released. Beta versions and dates are nice too. If there is a timed release schedule, tell me that.
Keep the information up-to-date. I shouldn’t have to wonder if your software is going to work under OS X 10.5 or RHEL 5, or if your plugin will work under the latest version of Drupal/Django/Moodle/MySQL/Joomla/Firefox…
BONUS: a very simple architectural drawing that shows me exactly what components make up the whole. The one for CouchDB is as good as any I’ve ever seen (assuming it’s accurate).
BONUS: if screenshots are applicable, use them. They communicate a million times more information using a million times less real estate and bandwidth. They can communicate things you didn’t even know you were communicating. Of course, that could be good or bad, but it keeps you honest, and customers like that :-)

For kicks, here are a few things I see sometimes on project web sites that I wish they *wouldn’t* do:

DON’T require me to understand how something like Trac or some other tool works in order to get at the information about your software project. Navigation should not assume I’m a developer, it should assume I’m a prospective user who will leave if they can’t read the menu. If you want to use a project management tool to do your work, more power to you, but as a prospective customer, it’s none of my business — don’t drag me into your personal hell! I just want the software!
DON’T be satisfied with the Sourceforge page as your project’s “homepage”. The problem with doing that is twofold: first, Sourceforge kinda sucks, and occasionally becomes unusable. Second, it doesn’t provide a simple way for you to give me information, nor a simple way for me to find it even if you produce said information using their tools. Also, it’s bad form. If you haven’t committed to the project enough to give it a proper site, well…
DON’T put some kind of “Coming Soon” page with a bunch of information with *NO DATE*, because I’m going to go ahead and assume that this thing is vaporware, and that the “coming soon” post is 3 years old. Nothing in this world is more annoying than time-sensitive information being plastered on a web site with no date.
DO NOT — I repeat — DO NOT force me to download a 20MB tarball to get at the documentation. That’s not how things work. I get to see what I’m downloading *before* I download it. You’ll save me some time, and save yourself some bandwidth, and you’ll have more accurate statistics about how many people download and use your software, because the numbers won’t be skewed by folks who were forced to download the package to get at the documentation.

All of that said, I probably won’t use CouchDB, even though I love their project’s site. Javascript makes my brain explode, so mixing them with something like a database, which to me is the digital embodiment of sanity itself, is… insane. But if you’re someone who can deal with this concoction, I encourage you to check out CouchDB — at the very least, you can figure out if it might be a fit for you without clicking from their home page a single time. That just rocks.

Robert Brewer: Writing High-Efficiency Large Python Systems--Lesson #1: Transactions in tests

Thu, 03 Jul 2008 22:02:59 +0000

Don't write your test suite to create and destroy databases for each run. Instead, make each test method start a transaction and roll it back. We just made that move at work on a DAL project, and the test suite went from 500+ seconds to run the whole thing down to around 100. It also allowed us to remove a lot of "undo" code in the tests.

This means ensuring your test helpers always connect to their databases on the same connection (transactions are connection-specific). If you're using a connection pool where leased conns are bound to each thread, this means rewriting tests that start new threads (or leaving them "the old way"; that is, create/drop). It also means that, rather than running slightly different .sql files per test or module, you instead have a base of data and allow each test to add other data as needed. If your rollbacks work, these can't pollute other tests.

Obviously, this is much harder if you're doing integration testing of sharded systems and the like. But for application logic, it'll save you a lot of headache to do this from the start.

Original post blogged on b2evolution.

Duncan McGreggor: Divmod Tech: Making the "Next Gen" Grade

noreply@blogger.com (Duncan McGreggor) — Thu, 03 Jul 2008 20:26:03 +0000

Last night, after I already posted the latest Twisted in the News, I came across another post that would have made the list had I found it sooner. However, this is a good opportunity to give it a little extra attention.

The title of the post is "Next Gen Web Dev: Playing with Python Twisted/Nevow/Athena" and I gotta say, that made my day :-) Between that post and Colin Alston's post that I mentioned in the News, Nevow had a good week. And people are appreciating it for the right reasons. It may not be the easiest web framework to use and certainly not the best documented, but when you need the flexibility to interact with your (Twisted) web server in particular ways as well as benefit from the functionality that COMET provides, Nevow comes out shining.

It's also refreshing to see new developers entering the community who not only see the potential of these tools (designed with that potential in mind) but are capable of taking advantage of it immediately. If nothing else, the author of that post has motivated me to finally merge the Athena tutorial to trunk in order to bring the publicly available and published content in sync with the new code that's in the branch.

Update: Along similar lines, but with more details, Tristan has provided an excellent write-up for this motivation to use Twisted/Nevow/Axiom/Mantissa. Be sure to check it out!

Atul Varma: Running C and Python Code on The Web

Thu, 03 Jul 2008 17:56:38 +0000

Last week, Scott Petersen from Adobe gave a talk at Mozilla on a toolchain he’s been creating—soon to be open-sourced—that allows C code to be targeted to the Tamarin virtual machine. Aside from being a really interesting piece of technology, I thought its implications for the web were pretty impressive.

Before reading this post, readers who aren’t familiar with Tamarin may want to read Frank Hecker’s excellent Adobe, Mozilla, and Tamarin post from 2006 for some background on its goals and why it’s relevant to Mozilla and the open-source community in general.

If I followed his presentation right, Petersen’s toolchain works something like this:

A special version of the GNU C Compiler—possibly llvm-gcc—compiles C code into instructions for the Low Level Virtual Machine.
The LLVM instructions are converted into opcodes for a custom Virtual Machine that runs in ActionScript, a variant of ECMAScript and sibling of JavaScript.
The ActionScript is automatically compiled into Tamarin bytecode by Adobe Flash, which may be further compiled into native machine language by Tamarin’s Just-in-Time (JIT) compiler.

The toolchain includes lots of other details, such as a custom POSIX system call API and a C multimedia library that provides access to Flash. And there’s some things that Petersen had to add to Tamarin, such as a native byte array that maps directly to RAM, thereby allowing the VM’s “emulation” of memory to have only a minor overhead over the real thing.

The end result is the ability to run a wide variety of existing C code in Flash at acceptable speeds. Petersen demonstrated a version of Quake running in a Flash app, as well as a C-based Nintendo emulator running Zelda; both were eminently playable, and included sound effects and music.

So, once Petersen’s modifications to Tamarin make their way into the next version of Adobe Flash, we can expect to see older commercial games running in the browser. Even more impressive, though, is the sheer volume of existing code that can be made to run inside the browser: Petersen showed us the C-compiled versions of Lua, Ruby, Perl, and Python all running on the web in secure Flash sandboxes.

What this means for Python

The potential implications this has for Python are particularly interesting to me. The ability to run Python on the web is exciting, to say the least; also interesting is the fact that by sandboxing CPython in a virtual machine, we solve a lot of the security issues that currently face the language when it comes to running untrusted code.

Petersen’s work also resonates with a few goals of another project called PyPy. I’m going to try to explain the idea behind PyPy in a later post; for the time being, the slides from my April 2007 ChiPy presentation on PyPy may serve as a passable introduction.

In a nutshell, the difference in mindset between PyPy and Petersen’s work is that the former is radically innovative in scope and mission, while the latter is pragmatic. PyPy’s goal is essentially to move the canonical implementation of Python from C to Python itself, and then use a pluggable toolchain to translate the Python interpreter to any platform with a configurable set of language and implementation features. In one fell swoop, this modularizes the composition of the Python interpreter in such a way that innovating and maintaining different ports and variants of Python like IronPython, Jython, and Stackless no longer requires either writing an entire copy of the same interpreter in a different language or branching the CPython source code and making pervasive changes to it.

Rather than focusing on innovation, Petersen’s work focuses on code reuse. Instead of moving a canonical interpreter implementation from C to a dynamic language, his strategy is to simply compile the existing C code to run in a virtual machine that’s implemented in a dynamic language. Both approaches aim to obviate the necessity of “ports” of interpreters to different platforms, and as such their purposes intersect at a common subset of functionality. But Petersen’s work can’t be used to facilitate the innovation of the Python language and its implementation, while PyPy offers few or no tools to reuse existing non-Python code. Perhaps it’s possible to combine the best of both worlds by taking PyPy’s generated C interpreter and using Petersen’s toolchain to allow it to be usable on the web and other places that Tamarin runs.

What this means for the Open Web

To be honest, I’m not quite sure where the dividing line is between what of Petersen’s work is Flash-specific and what can be reused to benefit the Open Web. Since ActionScript is a sibling language to JavaScript, it’s possible that the custom VM he created can be run in a browser with relatively few modifications—albeit much more slowly in Firefox at the time being, since SpiderMonkey-Tamarin integration is not yet complete. Once that’s further along, though, I imagine it should be possible to create C “libraries” that can be used in the toolchain to allow sandboxed C code to interact with web pages rather than Flash apps. Should this be feasible, I think it will possibly be the ultimate in a relatively recent string of next-generation Javascript virtual machines that allow existing code to run safely in browsers.

Also, in the context of the web, download size is a significant concern because applications are essentially streamed to clients. While Petersen’s toolchain means that it’s possible to instantly inherit most of CPython’s benefits on the web, it also means that we get all of its flaws along with it—such as the fact that the standard CPython distribution is a few megabytes large. But there’s ways to get around this.

In any case, I’m really excited to see how both Petersen’s work and PyPy proceed. I just hope I haven’t mis-represented either one of them here due to a lack of understanding; I’ll try to correct this blog post as I become aware of my mistakes.

Wybiral: Fresh new look

noreply@blogger.com (Wybiral) — Thu, 03 Jul 2008 16:56:43 +0000

I gave Challenge-You! a facelift this week to try to improve some of the design flaws in the original layout. Design is hard! I'm still looking for a CSS/HTML expert to help come up with and implement an interesting layout for the site (leave me feedback on the site if you're interested). I've improved some of the caching for the statistics and am working towards extending some of the core functionality (including branching out into other types of challenges).

I've also learned a valuable lesson in Python design... Be careful with decorators. Sometimes it's better to just wrap your function calls. I ran into some issues with needing some of the functions for alternate purposes, but they had been decorated for a specific purpose which resulted in lots of rewriting. :( But, lesson learned...

Corey Goldberg: Searching For Your Open Source Code

noreply@blogger.com (Corey Goldberg) — Thu, 03 Jul 2008 14:09:43 +0000

One cool thing about developing free open source software is seeing where your code ends up. It is satisfying to know that bits of code you wrote end up in other projects and code bases.

There are several online services that index source code from various places like websites and online svn/cvs repositories. It makes searching for my code a breeze. This is also useful for looking up examples of code I have written in the past that I want to copy from.

Google Code Search:
Search public source code for: "corey goldberg"

Koders Search:
Search public source code for: "corey goldberg"

Krugle Code Search:
Search public source code for: "corey goldberg"

Christopher Lenz: The Truth About Unicode In Python

Thu, 03 Jul 2008 13:32:31 +0000

The unicode support in Python is generally considered to be pretty good. And in comparison to many other languages, it's good indeed.

But compared to what is provided by the International Components for Unicode (ICU) project, there's also a lot missing, including collation, special case conversions, regular expressions, text segmentation, and bidirectional text handling. Not to mention extensive support for locale-specific formatting of dates and numbers and time calculations with different calendars.

Basically what Python does provide out of the box is “only” encoding/decoding, normalization, and some other bits such as simple case conversion and splitting on whitespace. It's the absolute minimum you need to do anything useful with unicode, but often not enough to build truly internationalized applications. (Fortunately, most applications get away without true internationalization.)

In this post, I'm going to talk about a couple of the problems with unicode in Python. Please note that this is not intended as a criticism of Python's unicode support or the people who designed and implemented it. Most of those people probably know a whole lot more about unicode than I do, and the limitations discussed here are the result of a pragmatic approach to implementing unicode support, rather than due to a lack of knowledge.

read on …

Arthur Koziel: yml2tex presentation from UAS Dortmund

Thu, 03 Jul 2008 13:11:46 +0000

I had the chance to give a small talk at the UAS Dortmund and introduce the yml2tex Python script to generate LaTeX Beamer presentations out of YAML files.

Although the presentation is in German, I think it might help to better understand the syntax by seeing a practical example.

The whole presentation with all the files needed to generate it can be downloaded in a ZIP file.

Andrew Dalke: HAKMEM 169 and other popcount implementations

Thu, 03 Jul 2008 12:00:00 +0000

This is part 6 of an on-going series dealing with molecular fingerprints:

Molecular fingerprints, background
Generating molecular fingerprints with OpenBabel
Computing Tanimoto scores, quickly
Block Tanimoto calculations
Faster block Tanimoto calculations
HAKMEM 169 and other popcount implementations
Testing the bitslice algorithm for popcount

Comments?

HAKMEM

Many other people have worked on doing high speed population counts. In 1972 the AI Laboratory at MIT published "HAKMEM", or "Artificial Intelligence Memo No. 239". It is a collection of algorithms, tricks, observations, and unsolved but interesting problems. One of which is to "Solve Go." Item 169 is a PDP-6/10 assembly way to count the number of ones in a word. It was called "count ones" and is now called popcount.

I first came across HAKMEM Item 169 in 2005 when I last looked at doing fast Tanimoto calculations. I tried it and a few other solutions but concluded that using special Altivec instructions were the fastest solution for my PowerPC-based PowerBook. I don't have a PowerPC any more. I bought a MacBook Pro last year with a 2.33 GHz Intel Core 2 Duo. I've been looking at the MMX instructions quizzically trying to figure out if there's an equivalent solution. I am not an assembly programmer.

The best I could find was an AMD "x86 Code Optimization Guide". It gives a 32-bit integer version using normal Intel assembly and a 64-bit version using MMX which "can do popcounts about twice as fast as the integer version (for an identical number of bits)."

The newer guide (I misplaced the URL) says to use the popcount instruction in SSE4a.

Other popcount references

Computing popcount is apparently very widely used. You can find many people have talked about it:

Bart Massey:

popcount_naive: 1.6666e+07 iters in 688 msecs for 41.28 nsecs/iter
popcount_8: 1e+08 iters in 995 msecs for 9.95 nsecs/iter
popcount_6: 2e+08 iters in 1725 msecs for 8.625 nsecs/iter
popcount_hakmem: 2e+08 iters in 1411 msecs for 7.055 nsecs/iter
popcount_keane: 1e+09 iters in 6566 msecs for 6.566 nsecs/iter
popcount_3: 1e+09 iters in 6270 msecs for 6.27 nsecs/iter
popcount_2: 1e+09 iters in 5658 msecs for 5.658 nsecs/iter
popcount_mult: 1e+09 iters in 4169 msecs for 4.169 nsecs/iter
  ...
At the suggestion of Bill Trost, I added 8 and 16-bit table-driven
popcount implementations. These perform the fastest in the benchmark
test-stand, at about 3ns/iteration. They're about the same speed, so
one would obviously use the 8-bit version.

Majek has pointers to code. That's where I found the AMD optimization guide link.
Lauradoux Cédric's Hamming weight page has an overview of the topic. He reports:

Algorithms Throughput

Naive 29.97 MB/s

Kernighan 33.34 MB/s

Tree 1 122.97 MB/s

Tree 2 130.60 MB/s

Tree 3 125.57 MB/s

Table 8 bits 200.94 MB/s

Table 16 bits 244.64 MB/s

and nicely summarizes why few people use the fastest solution:
The fastest solutions use pre-computation. However, you will never find this two implementations in current software product (X11, linux kernel. . . ). The reason of that is simple: the programmers are reluctant to use solutions implying a big amount of memory since it can affect the portability of the applications and their performance. As a consequence, the algorithms based on tree are the most popular.
There was a reddit thread in 2007 titled: Popcount - the "NSA instruction" missing from modern CPUs (or at least their datasheets.). In it, a1k0n posted a link to a popcount benchmark he wrote. After tweaking the 8-bit version slightly for better performance I got:
```
FreeBSD version 1   :  3784823 us; cnt=32511665
FreeBSD version 2   :  3421292 us; cnt=32511665
16-bit LUT          :  2068947 us; cnt=32511665
8-bit LUT           :  3632031 us; cnt=32511665
8x shift and add    : 21401951 us; cnt=32511665
32x shift and add   : 19419464 us; cnt=32511665
```
and he also found that lookups were the fastest solution.
There's a gcc bug report from April 2008 showing that that __builtin_popcount isn't even as fast as other bit twiddling solutions. Joseph Zbiciak attached a benchmark. On my machine it shows that gcc's builtin solution is about 1/3rd the speed of a better one:
```
popcount64_1 = 4114 clocks   gcc's __builtin_popcount
popcount64_2 = 1881 clocks 
popcount64_3 = 1627 clocks
popcount64_4 = 1381 clocks
```
There's also a number of popcount algorithms documented by Sean Eron Anderson including descriptions and further links.
Cédric in private mail pointed out a nice collection of popcount implementations from 1998/1999 including a cute 11 bit lookup table and an implementation of Robert Harley's bit slice solution, which is great when doing popcount across several words at a time.
Finally, you can fetch the GMP package. It contains hand-written assembly implementations for many different architectures. The Intel ones are based on bit twiddling and do not use a lookup table.

Algorithms	Throughput
Naive	29.97 MB/s
Kernighan	33.34 MB/s
Tree 1	122.97 MB/s
Tree 2	130.60 MB/s
Tree 3	125.57 MB/s
Table 8 bits	200.94 MB/s
Table 16 bits	244.64 MB/s

Benchmarking popcount in GMP

Interesting. Since I have gmpy installed on this machine I can figure out just how good gmp's hand-written bit twiddling actually is. Here's what I did, with commentary inserted.

# Read the entire data file into memory
>>> s = open("nci.binary_fp").read()
>>> len(s)
155667200

# Convert each character into hex

>>> s = s.encode("hex")

# Convert the hex number into a gmp integer, using base 16

>>> import gmpy
>>> n = gmpy.mpz(s, 16)

# Do the popcount

>>> gmpy.popcount(n)
136963400

# How long did it take?

>>> import time
>>> t1 = time.time(); gmpy.popcount(n); t2 = time.time()  
136963400
>>> t2-t1
0.36236190795898438
>>> print (155667200/256)*(t2-t1), "fingerprints per second"
220343.217182 fingerprints per second

# Verify that I got the right answer by using an 8-bit lookup table

>>> bitcounts = (
... 0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4,1,2,2,3,2,3,3,4,2,3,3,4,3,4,4,5,
... 1,2,2,3,2,3,3,4,2,3,3,4,3,4,4,5,2,3,3,4,3,4,4,5,3,4,4,5,4,5,5,6,
... 1,2,2,3,2,3,3,4,2,3,3,4,3,4,4,5,2,3,3,4,3,4,4,5,3,4,4,5,4,5,5,6,
... 2,3,3,4,3,4,4,5,3,4,4,5,4,5,5,6,3,4,4,5,4,5,5,6,4,5,5,6,5,6,6,7,
... 1,2,2,3,2,3,3,4,2,3,3,4,3,4,4,5,2,3,3,4,3,4,4,5,3,4,4,5,4,5,5,6,
... 2,3,3,4,3,4,4,5,3,4,4,5,4,5,5,6,3,4,4,5,4,5,5,6,4,5,5,6,5,6,6,7,
... 2,3,3,4,3,4,4,5,3,4,4,5,4,5,5,6,3,4,4,5,4,5,5,6,4,5,5,6,5,6,6,7,
... 3,4,4,5,4,5,5,6,4,5,5,6,5,6,6,7,4,5,5,6,5,6,6,7,5,6,6,7,6,7,7,8,)
>>> bitcount_table = dict((chr(i), count) for (i,count) in enumerate(bitcounts))

# Double-check the table creation -- this should be 2

>>> bitcount_table["A"]
2

# Do the full popcount; this will take a while

>>> sum(bitcount_table[c] for c in s)
136963400
>>>

GMP takes about 0.36 seconds to compute one popcount. My C code before loop unwinding using the 16-bit lookup table took 0.41 seconds to compute two popcounts, or about 0.2 seconds for one popcount. This is in very good agreement with the numbers from a1k0n's benchmark (3.4 seconds vs. 2.1 seconds).

Another popcount benchmark

So many different algorithms and the benchmarks aren't compatible enough that I could compare them. I decided to expand a1k0n's benchmark and include a few other implementations. The result is popcnt.c. Yes, I didn't even change the name. It implements quite a few algorithms but it's still incomplete. Anyone want to contribute assembly versions using MMX instructions?

Here's what my laptop reports. It's a 2.33 GHz Intel Core 2 Duo and I compiled with gcc 4.0.1:

% cc popcnt.c -O3
% ./a.out
FreeBSD version 1   :  3828096 us; cnt=32511665
FreeBSD version 2   :  3474751 us; cnt=32511665
16-bit LUT          :  2095066 us; cnt=32511665
8-bit LUT           :  5329692 us; cnt=32511665
8-bit LUT v2        :  3691475 us; cnt=32511665
8x shift and add    : 21675609 us; cnt=32511665
32x shift and add   : 19687827 us; cnt=32511665
Wikipedia #2        :  5264710 us; cnt=32511665
Wikipedia #3        :  5037636 us; cnt=32511665
gcc popcount        :  6088138 us; cnt=32511665
gcc popcountll      :  7548551 us; cnt=32511665
naive               : 23647023 us; cnt=32511665
Wegner/Kernigan     : 15026099 us; cnt=32511665
Anderson            : 63285941 us; cnt=32511665
HAKMEM 169/X11      :  4205008 us; cnt=32511655

The fastest implementation is still the 16-bit lookup table, at 40% faster than that fastest bit-twiddling code. The "8-bit LUT" implementation is by a1k0n. I noticed it was a lot slower than my code so I investigated. The only difference was the order of calculations for a simple addition. The "8-bit LUT v2" is my version. It's interesting to see that that minor change causes a major timing difference. And HAKMEM is holding its own; it's even better than the Wikipedia ones, which I didn't expect.

A Core 2 Duo is a 64-bit processor. I can ask gcc to compile for 64-bits and see if that makes a difference.

% cc popcnt.c -O3 -m64
% ./a.out
FreeBSD version 1   :  3882201 us; cnt=32511665
FreeBSD version 2   :  3382603 us; cnt=32511665
16-bit LUT          :  1562167 us; cnt=32511665
8-bit LUT           :  2013767 us; cnt=32511665
8-bit LUT v2        :  2054639 us; cnt=32511665
8x shift and add    : 21679291 us; cnt=32511665
32x shift and add   : 20050205 us; cnt=32511665
Wikipedia #2        :  2113876 us; cnt=32511665
Wikipedia #3        :  2102645 us; cnt=32511665
gcc popcount        :  8081774 us; cnt=32511665
gcc popcountll      :  4217908 us; cnt=32511665
naive               : 27717913 us; cnt=32511665
Wegner/Kernigan     : 16618378 us; cnt=32511665
Anderson            : 12300547 us; cnt=32511665
HAKMEM 169/X11      :  5643968 us; cnt=32511655

It does, for some things. The lookup tables are faster (and the 8-bit counts are now the same), and the 64-bit algorithms (Wikipedia, gcc popcountll and Anderson) are much happier. The parallel algorithm used by the Wikipedia implementations are almost as fast as the 8-bit lookup table and only 33% slower than the 16-bit lookup table. While HAKMEM got 25% slower.

Is there any way I can ask gcc to use the 64-bit instructions but still talk to a Python which was compiled for 32 bits?

Profile-directed optimizations with gcc

I recently learned how to do profile-directed optimizations with gcc using the -fprofile-arcs and -fbranch-probabilities options. Sometimes the compiler needs to make a guess on which assembly code is best for a given circumstance. For instance, if a branch for an 'if' statement is usually taken then it might provide hints to the processor pipline to optimistically assume the branch will be taken. It can't guess correctly all the time. Profile-directed optimizations works by asking gcc during compilation to instrument the program, running the program to get timings on how the code is actually used, saving the results to a file, then recompiling while asking gcc to use the instrumentation data.

Is it useful for my case?

     compile with instrumentation - these will be slower than usual!
% cc popcnt.c -O3 -fprofile-arcs
% ./a.out
FreeBSD version 1   :  4304773 us; cnt=32511665
FreeBSD version 2   :  4062710 us; cnt=32511665
16-bit LUT          :  2835959 us; cnt=32511665
8-bit LUT           :  5344514 us; cnt=32511665
8-bit LUT v2        :  4535290 us; cnt=32511665
8x shift and add    : 22246178 us; cnt=32511665
32x shift and add   : 19915850 us; cnt=32511665
Wikipedia #2        :  5626697 us; cnt=32511665
Wikipedia #3        :  5318770 us; cnt=32511665
gcc popcount        :  7005410 us; cnt=32511665
gcc popcountll      :  8173944 us; cnt=32511665
naive               : 62215630 us; cnt=32511665
Wegner/Kernigan     : 34541338 us; cnt=32511665
Anderson            : 65256755 us; cnt=32511665
HAKMEM 169/X11      :  4727297 us; cnt=32511665


      See the profile data?
% ls -l popcnt.*
-rw-r--r--   1 dalke  staff  9177 Jul  3 05:01 popcnt.c
-rw-r--r--   1 dalke  staff  1816 Jul  3 04:43 popcnt.gcda
-rw-r--r--   1 dalke  staff  7872 Jul  3 04:43 popcnt.gcno

      recompile using the profile data
% cc popcnt.c -O3 -fbranch-probabilities
% ./a.out
FreeBSD version 1   :  3815036 us; cnt=32511665
FreeBSD version 2   :  3477915 us; cnt=32511665
16-bit LUT          :  2251514 us; cnt=32511665
8-bit LUT           :  5291010 us; cnt=32511665
8-bit LUT v2        :  3936142 us; cnt=32511665
8x shift and add    : 21715704 us; cnt=32511665
32x shift and add   : 19708580 us; cnt=32511665
Wikipedia #2        :  5273262 us; cnt=32511665
Wikipedia #3        :  5074212 us; cnt=32511665
gcc popcount        :  6431689 us; cnt=32511665
gcc popcountll      :  7562124 us; cnt=32511665
naive               : 28044028 us; cnt=32511665
Wegner/Kernigan     : 15280455 us; cnt=32511665
Anderson            : 63431368 us; cnt=32511665
HAKMEM 169/X11      :  4209093 us; cnt=32511665

No significant differences over the normal 32-bit code. Nor did I see any real differences if I compiled with the architecture-specific flags:

cc popcnt.c -O3 -march=nocona

Domain applicability

I expect people will come across this page when looking for popcount implementations, see my numbers, and end up using the 16-bit lookup table. That might be the right solution but bear in mind domain applicability. I plan to compute popcount over megabytes. Most people, like the proverbial implementation in X11, work on only a few words.

It takes time to bring the lookup table into memory. On modern machines it's something like tens of instructions. If you're only doing a few popcounts, or a lot of popcounts but not all the time, then it's probably best to use bit-twiddling. It's easier to implement and for that case likely faster.

Dalius Dobravolskas: EuroPython 2008 - June 6th

Thu, 03 Jul 2008 08:17:45 +0000

This year EuroPython begins on June 7th... but June 6th is public holiday in Lithuania. That means if you will arrive at EuroPython on June 6th or earlier don't miss some public events.

In 1253 on June 6th the first and the only Lithuanian king was crowned. What kind of event you should expect:

In S. Daukantas square Lithuanian state and historic flags will be hoisted. Lithuanian president will give the speech.
Mass will be held in Vilnius Cathedral.
The live archeology days will happen in Kernavė. That's not in Vilnius so you will need to get there somehow. Here and here you can find some photos from this event in the past.
In Sereikiškių park international folkloric town "Baltica-2008" will be build. You could attend concerts here, watch handicraft and traditions demonstrations, taste national food.
Castle festival masters and musicians town will be build in Museum of theater, music and cinema.

Here is map with these events locations + EuroPython location:

View Larger Map

Link to map: http://maps.google.com/maps/ms?ie=UTF8&hl=en&msa=0&msid=100008304315880675937.0004511c399027fb04a67&ll=54.740875,25.214996&spn=0.349611,0.821228&z=11&iwloc=0004511c8bf9853680e3f

I will update this entry when I will have more information.

Dalius Dobravolskas: EuroPython 2008 - July 6th

Thu, 03 Jul 2008 08:17:45 +0000

This year EuroPython begins on July 7th... but July 6th is public holiday in Lithuania. That means if you will arrive at EuroPython on July 6th or earlier don't miss some public events.

In 1253 on July 6th the first and the only Lithuanian king was crowned. What kind of event you should expect:

In S. Daukantas square Lithuanian state and historic flags will be hoisted. Lithuanian president will give the speech.
Mass will be held in Vilnius Cathedral.
The live archeology days will happen in Kernavė. That's not in Vilnius so you will need to get there somehow. Here and here you can find some photos from this event in the past. This event happens from July 5th to July 7th. I have even find the whole program: http://www.eb.lt/en.php3?vid=67&id=3860
In Sereikiškių park international folkloric town "Baltica-2008" will be build. You could attend concerts here, watch handicraft and traditions demonstrations, taste national food.
Castle festival masters and musicians town will be build in Museum of theater, music and cinema.

Here is map with these events locations + EuroPython location:

View Larger Map

Link to map: http://maps.google.com/maps/ms?ie=UTF8&hl=en&msa=0&msid=100008304315880675937.0004511c399027fb04a67&ll=54.740875,25.214996&spn=0.349611,0.821228&z=11&iwloc=0004511c8bf9853680e3f

I will update this entry when I will have more information.

Sean McGrath: Economic necessity is the mother of software reuse

noreply@blogger.com (Sean) — Thu, 03 Jul 2008 08:01:00 +0000

Even recessions have up-sides : Economic necessity is the mother of software reuse

Corey Goldberg: Twitter API vs. Yahoo Web Services - Performance and Reliability

noreply@blogger.com (Corey Goldberg) — Wed, 02 Jul 2008 19:48:14 +0000

I've noticed the Twitter API is very slow and unreliable. Requests often take 10+ seconds to return. Sometimes they timeout with no response at all.

I put together some monitoring tools to see exactly how slow and unreliable the API really is (when it is even available!). This is obviously not a very scientific test; just a ballpark idea of performance. The monitoring tools are built with Python and RRDTool. To start with, I used a modified version of my rrdpy suite of tools (http://code.google.com/p/rrdpy).

I ran a test for 2 hours, hitting the API's Test Method every 30 seconds. The tools generated the following time-series graph of server response times:

The average response was 2.53 seconds, and we see a *lot* of variance in the data samples.

In comparison, here is the same test run against the Yahoo Search REST API:

Notice the small variance and the average server response time of 0.186 secs!

I would be interested in running these tests again in a few months to see if Twitter's API has become faster and more reliable.

Paulo Nuin: Bioinformatics career survey

Wed, 02 Jul 2008 16:48:07 +0000

Via Bioinformatics Zen:

Loading…

IronPython Url's: Programming Languages @ PDC 08

noreply@blogger.com (Fuzzyman) — Tue, 01 Jul 2008 22:13:10 +0000

Harry Pierson (IronPython PM for Microsoft) has posted some of the programming language sessions that will be part of the PDC 08 conference.

Naturally this includes a session on dynamic languages.

Deep Dive: Dynamic Languages in .NET
The CLR has great support for dynamic languages like IronPython. Learn how the new Dynamic Language Runtime (DLR) adds a shared dynamic type system, a standard hosting model, and support for generating fast dynamic code. Hear how these features enable languages that use the DLR to share code with other dynamic and static languages like VB.NET and C#.

Programming Languages @ PDC 08

For programming language geeks it looks like there is a lot to looks forward to at PDC, with sessions on F#, the future directions of both C# and VB.NET (both of which will include support for better working with dynamic languages), and C++.

IronPython Url's: Python Generation using the Code Dom

noreply@blogger.com (Fuzzyman) — Tue, 01 Jul 2008 22:12:22 +0000

IronPython 1 has a CodeDom provider. This means that IronPython Studio can use it to generate Python code from the Windows Forms and WPF designers, and even generate executable files from Python projects.

This is one feature that hasn't yet made it into IronPython 2, but there are strange rumblings from the IronPython team that it may just make it.

In the meantime, 'tech-michael' has a blog entry showing how to use IronPython CodeDom to generate Python code from metadata.

Python Generation using the Code Dom

Base-Art: Hiring at Flumotion

Tue, 01 Jul 2008 20:13:50 +0000

So Flumotion, sister company of Fluendo Embedded where I work, is looking for Python hackers. Thomas will be at EuroPython, so if you are interested in the available positions, be sure to get in touch with him!

Working at Fluendo is quite nice, conditions are great, people are nice and smart :) The office is growing quickly and Barcelona is at its awesomeness level!

So if you are familiar with Twisted, Python, GStreamer and dealing with high-traffic streaming (peaks at 6Gb/s these days), be sure to contact Thomas, don't be shy! If you are not familiar with these technologies but if you learn fast, contact Thomas anyway :)

Sylvain Hellegouarch: CherryPy 3.1 has been released

nospam@example.com (Sylvain Hellegouarch) — Tue, 01 Jul 2008 19:46:28 +0000

Yesterday Robert Brewer released version 3.1 of the CherryPy product and I think this calls for a hurray! For those who've been using 3.0 they'll be happy to know that the upgrade will be rather smooth and straightforward. The main API changes have taken place on the engine thanks to the new process bus but the rest of the API is pretty much identical. Overall the fantastic work Robert and many contributors have done was to fix remaining and new bugs making this release the most stable and high-performance of the CherryPy releases. Get it while it's hot.