Python Desktop Server Weblog 2003/10

Bug in the preview feature

There is a bug in the preview server that triggers a Servererror when you try to use the preview. This bug is fixed in the CVS version. I will put out a new version 0.6.3 within the next days.

Update: the 0.6.3 release will go out early next week (I am waiting for an updated japanese language file).

New Beta 0.6.2: several bugfixes

Top 10 Things Likely To Be Overheard From A Klingon Programmer

Hosting options for PyDS

The Hindu : Personal weblog-cum-news aggregation tool

XML.com: The Atom API [Oct. 15, 2003]

I am not very happy with this article. Reasons:

The Atom API was designed because the MetaWeblog API proved that RPC-based APIs were simply the wrong solution for this problem.

Wrong. This is not the case. What was discovered was that if people start out with a data representation and stick heavily to this data representation, a REST style API is much more fitting than a XMLRPC API. But if you do your data design in the terms of Collections, Structures, simple data types first - an XMLRPC API is still the way to go.

There is an especially annoying bit in the article: the what goes over the wire bit. In the context of XMLRPC calls, the wire format is irrelevant. This was a constant point of confusion in the wiki for quite some time, as document centric view and functionality (API) centric view are quite different in this area. If you work with abstract data models and distributed functionality, you don't necessarily have to care for the wire format. It would be quite easy to get the same results (functionality wise) as Atom by specifying data models (in terms of collections, structures, simple data types) and a classic API specifications with XMLRPC. There is no hidden law that forces one to solve this in the document centric way that Atom has chosen. I am not against the document centric way - it has it's benefits. But the underlying tone of the article is that XMLRPC interfaces and data structures are inferior to the Atom approach, especially because of the wire format of XMLRPC - and that hidden statement is plain ridiculous.

Well-defined data model -- with schemas and everything!

Wrong. Nowhere in the Atom spec there is a data model defined. What is defined is a data representation. And specifications of this representation. And APIs that build on this representation. This is not to downplay the accomplishment - to create a usefull data representation with spec, API and all is a very big step in itself. But it isn't a data model. It can be mirrored in a data model - or can reflect with data models, sure.

Doc-literal style web services, not RPC

This wasn't a goal of Atom. It was an outcome of the discussion based on an adhoc spec and sample and on how to work with this. It was a direct outc ome of the fact that more people preferred to work document and representation based than data model based. Would the start have been an abstract data model, the outcome might have been different (not necessarily better!). Atom never had fixed goals in the early design phase, beside being something built by community to solve a defined problem area (and even that was shifting while discussing it). To state that getting rid of XMLRPC was a design goal of Atom is plain wrong.

Many of the stated design goals in that article were actually just outcomes of discussions. For example the taking full advantage of HTTP was absolutely not an easy accomplishment - there were lots of discussions around this, wether PUT and DELETE were acceptable or should be avoided! Even one of the first prototype implementations of the API didn't use PUT and DELETE!

Previous weblog services had no concept of API discovery.

Wrong. There are several takes at API discovery (telling the XMLRPC connector URL inside data representation formats like rendered HTML or RSS and by using introspection on those connectors. This was just in it's infancy, so don't expect many fledged out specs, though. It's in it's infancy with Atom, too. API discovery is never an easy thing, especially if you need extensibility in your API. The document centric approach in Atom does help here, as the API spec itself has much less weight on functionality (only providing some primitive operations) and much more weight on representation (by just pointing over to some big honking document telling how to parse what you get). This is one of the very nice points about the Atom spec in special and the document centric view in general.

Sorry, Mark, you won't gain good marks from me for this one. Drop out the XMLRPC bashing and the (historical wrong) hype, and the rest makes quite a good introduction into Atoms functionality and power. But with the first part attached, it makes bad feelings with people that like to follow different paths to implementations.

BTW: One problem with Atom isn't mentioned in the article at all. It has to do with the document centric view - because of this, you are more in a naked objects world. Your documents are your addressable objects. What you see is your object. There isn't much substructuring of data - if something has an URL, it's an object and it's one object. This is in direct conflict with microcontent ideas, as those are only parts of visible objects. Sure, URLs can have a fragment part and you can use that to address subparts of an object, but this is a flat namespace. To create hierarchic subdivided content objects based on this is a bit wacky. It can be done, for sure. And you can do it by only working on your base content and defining some pages as derived objects that just pull stuff from the central repository and not having their own actual functionality, so it's not a big problem. But it is a bit problematic - when are you talking about the visible object, the page, when about the template that renders this page and when are you talking about the data that makes up the page.

Wild headscratching going on ...

I don't know the solution to the following problem, so I thought I just ask you :

When I use Safari to work with a PyDS installation, everything works fine if it is my local machine with my local PyDS. It does work fine when accessing a PyDS on my work machine that's hidden behind two cascaded ProxyPass-Apache-VirtualServers (just virtual servers that pull their content via non-caching proxypass directives from a machine behind them).

It doesn't work fine when I access a PyDS installation at simon.bofh.ms - my main private webserver. The weird thing: the setup isn't actually really problematic, it's just a PyDS running on a different machine. No cache in between. No special hacks working. Just two machines, one the browser, the other the PyDS instance.

What happens is the following: from time to time I don't get the current content of a page, but an old one - even though pages are marked as Pragma: no-cache and the browser does say it doesn't cache them. But I still get those that old content. Most often this happens in the aggregator - I purge a feed and still see that feed listed in the channel overview, even though it's already empty (and a reload shows that it is empty).

This only happens with Safari and the new Omniweb (using the same WebKit as Safari), so I assume it must have something to do with Safari itself. But I absolutely don't understand what is going on ...

If I use Firebird instead of Safari, everything works.

Update: I just put my PyDS with the problem behind an Apache server, so it is used mostly like the one at work. The weird thing: while the one at work works fine, the one with the problem still has the problem. It caches, although there is no caching allowed and needed. Werid.

Another update: when changing the redirecting that's taking place from the intermediate HTML page to direct Location headers and 302 errors, the problem with Safari disapears. This is really weird stuff, but since it's gone for now, I don't really care that much

Little bug with PyDS and encoded entities

There is a small bug with HTML entities in PyDS: if you create postings with entities, they are stored fine. But if you edit this posting (or story or some other places where this problem exists), the entity get's decoded. This creates problems if you want to put in literal entities for example in source snippets that include HTML templates or stuff like that. For example &amp;lt; would be decoded one step with each edit, first being reduced to &lt; and then to <.

If you experience this problem, try the CVS version. This includes a patch for this problem (each textareas default value is now protected by a call to macros.quoteHTML).

New beta 0.6.1: small bugfixes to the 0.6.0 release

Just a small list of bugfixes to 0.6.0 that shown up after releasing that one. It's as allways - bugs only show up after the next release

Especially there was a weird behaviour in the enclosure handling in the aggregator, that's now fixed.

Call out for Participation

Not only software developers can help the project. Other people can do, too. Here are some fields where PyDS could need your help:

• Louis Feges started a first take at a PyDS Users Guide. I'm sure he could use some help there.

• HowTos and FAQs for PyDS typical problems are allways needed. If you did something cool with PyDS - why not tell others about it?

• PyDS still needs a nice looking logo. It should include the Python snake and maybe a desktop?

• If you speak some language fluently, help with translation of PyDS. Take the pyds-de.msgs and start from there, or use the following code line in a CVS tree to get a starting point (set XX to your language code):

  python make_msgs.py pyds XX >pyds-XX.msgs-new
  

• of course, if you have some hack for PyDS that can help others, or fixed a bug or added compatibility patches for some system, those are allways welcome!

If you think you want to help with one of those things, join the development list and tell us. We help you get started (or point you to others that took over the same task). Contributors will be named in the distributions README file, so fame and fortune are awaiting you!

Scripts to automatically install PyDS on Mac OS X

Ian Cardenas built some shellscripts to pull down the needed archives, unpack them and install PyDS (his scripts install to /usr/local/pyds) automatically. Currently it needs an installed wget - get that via fink or darwin ports if you don't have it already.

Hmm. Somebody up to the task to build Mac OS X installer packages for all components?

Updated windows installer available

RPMs for PyDS

Austin Acton just contacted me by eMail and told me that the Python Desktop Server will be in Mandrake 9.2 - so this actually is the first distribution that delivers it directly (Debian is ready, but the packages are not in the main distribution but on my own server in my own repository).

I don't know wether the packages work with other RPM based distributions. If you try them out and they work, just drop me a note and I compile a list in the installation instructions.

Feed Parser under Python-License

New Beta 0.6.0: lots of feature enhancements

The 0.6.0 Version for the Python Desktop Server is out! This release includes a very big list of changes, many enhancements, bugfixes and reworkings of internal stuff. This is a major upgrade, so expect things to change!

Changes in the Release include (for a full list read the changelog):

• PyDS now supports gzip encoding in the downstreaming process. This helps saving bandwidth of RSS feed providers, if their server supports gzip too!
• RSS Feeds are now supported more completely. From the rather unfunky but cool enclosure item element to the rather funky but still cool content:encoded element.
• You can not only fetch enclosures, you can produce them, too!
• PyDS now fully supports UTF-8 encoding, if you enable it in your config as your documentEncoding. This allows people to use e.g. japanese characters (there is even a first take at an japanese translation done by Yasushi Iwata).
• PyDS now has an upstream driver architecture. You no longer need a community server, but can upstream to a normal FTP server or even to a local path on your harddisc, if you choose so. Enhanced features like comments or trackback are still bound to the usage of a community server, though.
• there is now a preview server integrated, where you can see what you changed before you decide to go online (yes, you now can toggle online/offline usage in PyDS!) and upstream your work.
• Tool levels and Tool admin requirement on remote access can now be configured in the preferences. No need to hack the config any more for this.
• a new tool specially taylored to produce WebZines was added.
• PyDS now looks very nice in W3M
• ent:cloud and end:topic are supported in the RSS feed.
• lots and lots of inner changes, too much to list them all.

Since this version did change a lot, you need to be carefull on upgrade. Although it should go seemless with regard to your data, there are some places where it might throw exceptions. A good first take is to edit (just edit and save, no changes needed) the item that produces an exception (Up to now I only noticed this behaviour with the Wiki). And all templates changed, so you either have to use the new default templates and port your changes back or port the changes in the default templates forward to your templates.

Important: two support modules must be upgraded for this version:

• medusa must be upgraded to 0.5.4 because of an annoying bug in pre 0.5.4 releases with regard to request parsing
• docutils must be upgraded to 0.3, as that is what we now use (and docutils 0.2 isn't API compatible with 0.3, so PyDS will break if you still use docutils 0.2!)

Due to the change to docutils 0.3 there are bound to be minor problems with rendering. If you notice anything that rendered differently with the older PyDS, please give me the structured text to reproduce this problem.