Python Desktop Server Weblog 2004/12

a picture of myself

Münsterland.org

2004-12-27

Futures as a different kind of Promise

Futures are just Promises - I was reminded of this fact by the Alice Tour. So I just added futures to my lazy evaluation module. Futures are just promises that are not calculated when the result is needed, but that start calculating the value right along - in parallel to your main thread. So every future represents a thread that calculates some result. When you force the result of a future that isn't already done, your main thread will block until the future is done and then continue with either the value or the exception of the calculated future. It's a nice and clean way to do multithreading in a small scale:

from LazyEvaluation import *

# define the fibonacci function
def fib(n):
    
if n in (0,1): return 1
    
else: return fib(n-1) + fib(n-2)

# make fib a future generating function
ffib = future(fib)

# create some future promises
fib5 = ffib(5)
fib10 = ffib(10)
fib20 = ffib(20)
fib30 = ffib(30) # that one takes quite long

# calculate some fibonacci number in the foreground while
# other calculations proceed in the background
# (due to the GIL in Python this won't make use of multiple
# processors and so can be seen as a purely academic
# exercise!)
fib15 = fib(15)

# now print the values - this will force the values
# and will block until the value is available
print fib5
print fib10
print fib15
print fib20
print fib30
This post references topics: python
posted at 21:26:40    #
 
2004-12-25

Python 2.2 and extended slicing

Can it be that although extended slicing is supported in Python 2.2, sequences (lists, tuples, strings) only support simple slicings? This is rather weird - __getitem__ get's passed in a slice object in Python 2.2, but the builtin types don't accept those slice objects as index values - and they don't accept the extended slice syntax at all.

l = range(0,10)
print l[1:5]     # this works
print l[1:5:2]   # this barfs

Weird. Really weird. Makes life not easier with my LazyEvaluation stuff - I want to support Python 2.2 and so need to hack something up that will work with it. Looks like I will still have to overload __getslice__ and friends ...

This post references topics: python
posted at 12:03:28    #
 

Making the impossible possible

Lazy evaluation is a way to delay execution to some later point in time - you build up a promise to deliver the result, but you only deliver it when it really is needed. As long as it isn't needed, it isn't calculated. Lazy evaluation is usefull when you have complicated calculations whose result might not be needed at all - but you don't know up front.

Usually you do it by boxing the expression in some closure and calling that closure later on when you need the value. Easy as pie. But this has a problem: what if you have a data structure built by dictionaries, objects with methods, attribute access and stuff like that? Of course you can box the full expression - but that would require that you know the whole expression up front. What if the expression is built piece by piece over time?

Lazy languages push this concept over the border: all expressions are evaluated lazily. But this requires a new language - normal languages with strict evaluation don't allow grafting lazy evaluation onto them after the fact.

So I thought what is needed is something like faked lazy evaluation in Python. Yeah, that's it - it quacks like lazy evaluation, it smells like lazy evaluation and it mostly looks like lazy evaluation. So let's call it lazy evaluation Winky

Using it is easy:

from LazyEvaluation import *

def anton(a,b):
    
return a+b

f = lazy(anton)

print repr(f(5,6))
print f(5,6)

The first print will print a Promise() instance representation. The second print will print 11 - because print does str() on it's arguments and that will call the magic method __str__ and that one will first force the evaluation of the Promise() and then apply the original method.

I know I am crazy. And maybe all this is just stupid. But it was fun to hack. It makes heavy use of magic methods, meta classes, closures. Hey, if I already had Python 2.4 installed I would even have used decorators (it should fit into 2.4 quite fine - just use lazy as a decorator if you want to make one of your methods lazy evaluated). You have been warned. Winky

This post references topics: python
posted at 03:10:08    #
 
2004-12-12

Following the Active Storage Framework Development

There are currently the following resources to follow the development of the Active Storage Framework for Python:

If there is enough interest, I might add a mailing list. So far nobody mailed me about the project, so I think the interest is quite low at the moment Winky

Another note: the database currently is highly alpha. It happens allmost every day that I rewrite large chunks of code, throw out complete modules and change internal structure, change attribute types, invent new ones, throw out old ones. To make a long story short: if you follow the system, your snapshot of yesterday will be guaranteed to be outdated Winky

If I ever get it to some stable state (API wise), I will put out a 0.2 version and starting with that begin to slow down a bit. But as long as it is 0.1.0, it is supposed to be changed.

This post references topics: python
posted at 14:34:56    #
 
2004-12-08

PyDS Release 0.7.3 is out

I just released the release 0.7.3 of the Python Desktop Server. Lot's of changes:

  • several fixes and changes by Garth to the WikiTool
  • unnecessary locks removed to make it more responsive
  • tools to debug the locking
  • newMediaObject is supported in the MetaWeblogAPI (only images so far)
  • integration of the blogmarks and pictures RSS feeds into the main RSS feed
  • lot's of smaller changes
This post references topics: python python_desktop_server software
posted at 11:37:52    #
 
2004-12-07

Don't know what it says about me ...

... that the two python extensions my current pet-project Active Storage Framework for Python depends on are named psycopg and syck Winky

On another note: I now have development debian packages up and the installation via setup.py should actually work now. I currently still test directly in the source tree, that's why I never noticed it was broken ...

posted at 19:18:40    #
 
2004-12-05

syck - a really nice and fast YAML parser

While I worked on implementing database backup in YAML for the Active Storage Framework for Python, I played around a bit with the two existing YAML parsers for Python. pyYaml - we shouldn't speak about it. It's broken. But syck is cool - it's fast, it's fully featured and did I say it's fast? Boy is it fast ...

So now YAML is the format of choice in the Active Storage Framework for Python to get data in and get data out.

In general the Active Storage Framework for Python is hopping along nicely. If you looked at the README some days ago, you might want to look again - lot's of changes in the code, many features implemented. I am quite happy with the progress of the project. Even though such a coding frenzy doesn't do good for my back ... ":-/"

This post references topics: python
posted at 01:14:56    #
December 2004
MoTuWeThFrSaSu
   1 2 3 4 5
6 7 8 9101112
13141516171819
20212223242526
2728293031  
Nov
2004
 Jan
2005

This is the Python Desktop Server weblog.


(Donations will be used by the author to buy stuff, fullfill selfish wishes or do other silly recreational things. You have been warned.).
The PyDS is
OSI Certified Open Source Software

Python Powered

XML-Image

© 2004-2007, Georg Bauer