metajack - Latest Comments in The Problem With Django

Re: The Problem With Django

axquan — Thu, 12 Jun 2008 11:51:00 -0000

Great blog post. Short, simple and right to the point. Good work!

As to the discussion. From where I'm standing it seems like Django developers are hold off by a common disease among programmers - perfection. On one side it is very useful to have someone to look at every detail, but on the other hand, it sometimes gets excrutiantly slow to do anything at all.

Having that in mind I see only two directions in which Django team can go:
1) You remain status quo and pretend that nothing is wrong. You take your time and release 1.0 in a matter of 3 months perhaps.
2) You get over yourself, release 0.97 as it is now and gradually go for the 1.0 grand release.

It may seem like it's just pure nonsense to release just because you can, but if people are already using trunks in production as opposed to release versions, why not make it easier for those few which actually care and want to contribute documentation and add-ons to the project?

As it was said before, who knows when the goals for 1.0 will be achieved, it could be those 3 months or it could be as long as 6 months. And that in a software world is eternity. Think about it.

Cheers!

Re: The Problem With Django

newz2000 — Thu, 12 Jun 2008 07:55:00 -0000

Let me mention one problem I've run into recently with the lack of a recent release. Regarding documentation, I can't refer people to a feature in the documentation unless it's for 0.96 because the head documentation changes. So if I need to count on a feature being documented I have to use 0.96 and link to the documentation there.

I would really really love more frequent updates. I just can't run off of svn but would love some of the features.

There's another good reason to have frequent releases. It communicates to people who are evaluating Django that the project is alive and thriving and gets them excited to use it and join the community. This turns into a self feeding cycle where the buzz grows and grows.

Switch to a development model and toolset that facilities frequent releases. You can use bzr and keep different branches of your code locally (and bzr's shelf plugin for when you want to return to head for a little while and then resume working on your branch). This will enable you to create regularly scheduled releases and still work on tasks that take longer to accomplish than one release cycle allows.

Scheduling a release for every 4 months would give you three per year. The last two weeks could be for testing and code would be frozen.

(btw, you don't have to give up trac to use bzr, since bzr can be used as a frontend for svn)

Re: The Problem With Django

Manny — Thu, 12 Jun 2008 04:09:00 -0000

Much applause Jacob--your response is much more preferable than the defensive posturing we've seen from others.

Re: The Problem With Django

David Cramer — Thu, 12 Jun 2008 01:43:00 -0000

I have also run into the issue of releases not happening and myself doing an SVN update in order to try and solve an issue which was present in trunk, and then other issues arising because of more backwards incompatible changes. Yes, this is my fault. Yes, I could have wasted time reading over a bunch of changelogs. Yes, I would prefer more releases so I wouldn't have to use trunk on projects. Revision XYZ is not a proper versioning schema.

This is something I've argued for for quite a while. A great example is the "Queryset Refactor" branch which was in the works for many many months, and a lot of Django's "core" features are like this. There is a lot of functionality that truly is needed by people like myself, people who aren't writing very basic blog software, or who are writing software that needs to scale in very typical situations. While some of the stuff I/we do may be edge cases, I've found through discussion that a lot of it is quite common.

After a lot of experience with patching Django, I for one can say it is NOT fun having to branch off, and it's not something I want to ever do on any current or new projects. But when you have so many people willing to fix the current issues, and then very few who seem to be "allowed" to fix them, nothing ever happens. This presently has made me consider moving away from Django as we already have switched away from it's template engine, and the ORM limitations and lack of updates make it very difficult to complete some projects.

A lot of people may disagree with me, but that's their right. Django is a very nice framework, and I believe it's one of the better frameworks out there. Can it be used for every project? Not if you don't want to deal with headaches.

I for one will continue to use Django, on any project I can, as I very much enjoy using Python. But if a project needs to perform well, and I have the slightest doubt Django can't perform in that situation, I'm not even going to think of recommending it.

Re: The Problem With Django

casseen — Thu, 12 Jun 2008 00:35:00 -0000

I don't get all those complains over long release cycles. First of all it's stated that *after* 1.0 things will change. Secondly, developers agreed 1.0 will be the next release. So there just won't be any intermediate releases.

The big pro about open development is you can just kick in and devote your time to develop the project. You will have a voice, you can influence development and planning, and after all you can speed things up.

Constructive criticism is indispensable, it's the community's obligation. But it should not annoy you if things do not strictly follow your expectations.

Re: The Problem With Django

Jacob Kaplan-Moss — Wed, 11 Jun 2008 16:48:00 -0000

Jack: to a great extent, the problems you describe are my fault, and I apologize. Thanks for the constructive criticism.

I hope I can talk you into turning some of your energy towards helping us get 1.0 out as soon as possible. I've posted a proposed roadmap on django-dev (http://groups.google.com/gr... I'd especially like to see your feedback.

Re: The Problem With Django

Jenny Cowan — Wed, 11 Jun 2008 15:29:00 -0000

I thought the whole point of dynamic languages was a fast release schedule?

I mean, if I wanted slooooowwww I would look at java.

Come on django step up

Re: The Problem With Django

Kevin Teague — Wed, 11 Jun 2008 11:55:00 -0000

It is not good when people are releasing django add-on projects against "a recent trunk". Not all releases need to be stable and finished. Releases with names such as -dev, -svn-r***, -preview, -rc, -beta, and -alpha are possible. Stating that an add-on project depends upon "django-1.0-alpha1" is much better than "a recent trunk", because you won't break that add-on project's installation insructions and dependency information when trunk gets updated.

Ideally a Django add-on should be able to state in it's setup.py (or somewhere such as packages.cfg or versions.cfg) "install_requires = ['Django' == 'django-1.0-alpha1','django.contrib.comments' == '2.0','djano.someotheraddon',]".

Grok has been going through the teething process of refining it's release and dependency management in the last year or so, but it's getting quite slick (I would say it's dependency management is better than the recent dependency features added to Rails 2.1). With a Grok application, a release is just a list of versions of the specific packages it's composed from, e.g.:

http://grok.zope.org/releaseinfo/grok-0.12.1.cfg

http://grok.zope.org/releaseinfo/grok-0.11.cfg

This is useful, because if I have a Grok 0.11 app that I want to bring up to 0.12.1, I can just diff the two lists of package specifications and see which parts of the application have changed.

A Grok add-on can specify the version of Grok it requires, and you can also specify that you wish to upgrade a single package from the Grok release to a newer version. When managing a project's dependencies there is no difference between "the packages that compose a web framework", "the packages that are installed from reusable application add-ons" and "your own development packages". It's easy to update the "working set" of packages with a newer release of a specific package, regardless of what part of the stack that package came from. Specific packages can also specify that they do or do not work with older/newer versions of the packages that they depend upon.

If you are trying to get two add-ons with incompatible version dependencies to work together, it's a lot easier if you can see "add-on A requires version = 0.9 of package C" and "add-on B requires version 1.0 of package C". You know that you need to resolve in add-on A what is keeping it from working with version 1.0 of package C. Instead in Django you would have to do more digging when you hit version conflicts since you would be presented with only "add-on A requires a recent trunk" and "add-on B requires a recent trunk".

Re: The Problem With Django

Ungamedplayer — Wed, 11 Jun 2008 11:48:00 -0000

If anyone has made it reading this far, this article is/was written to capitalize on the recent "why don't you release" series of articles.

Most of this noise is not relevant to 90% of the sites.

If there are so many complainers, there is the legit process of forking this program. I guess that perhaps the complainers figure that maintaining a project is considerably tougher and complaining is just easier.

Re: The Problem With Django

Geoff — Wed, 11 Jun 2008 11:30:00 -0000

Maybe you should consider talking with other project people about how they organize releases. At the OpenCMS Summit at Yahoo last year, I was impressed by the way the Drupal and Joomla developers talked shop. There are certainly differences but the larger sense of open source community and good fellowship brought them together over shared issues, and my personal experience with the drupal community has been very positive. I'm not currently developing with Django, but I follow it since it does share a similar set of values as projects like drupal.

Re: The Problem With Django

victor — Wed, 11 Jun 2008 11:12:00 -0000

I feel the author. I really do. And sometimes, you just get tired of the Django philosophy. The author talked about patches, well some are not *officially* supported (will not be implemented, anyone?) by the framework...

Re: The Problem With Django

dude — Wed, 11 Jun 2008 10:27:00 -0000

Give up and learn Merb or Pylons or something. These Django guys don't know what they're doing. That's why I jumped ship after finishing my first Django project (almost 2 years ago, 0.95!!!).

Re: The Problem With Django

Wolfram Kriesing — Wed, 11 Jun 2008 10:27:00 -0000

I gave up pretty quickly to submit patches to django. They always got stalled and discussed to death, it just felt very unproductive.
Currently I run from one version and do workarounds in the code, I am tired of diving into django code and hoping that this patch may make it one day. I need to get things done.

It's just funny that it's expected that people run their productive apps from svn revisions but to mark one of those revisions with a version label seems a too high goal.
So the django user has to choose himself which is his a "1.0", delegating responsibilities.

I think django is great, it amazes me every time. But nobody is perfect :-)

Re: The Problem With Django

pkenjora — Wed, 11 Jun 2008 10:10:00 -0000

I think you hit it right on the nose. As a contributing Django developer I've patched several of my projects to fix database support as well as add some pretty powerful features. Unfortunately I've been way too busy to deliver the changes to trunk. Don't get me wrong, I've tried, but the process seems so very complex.

I think this issue occurs ore in Django because it has such a low barrier to entry. Django is NOT a framework for elitists, so maybe part of the problem is that the code delivery and patch process is way too confusing for novice open source contributors.

I say if you change anything, simplify the patch and code delivery. Then again maybe having it be tough keeps the bad code out? Its not an easy problem but from this post I take it that maybe a tutorial in delivering Django code to the trunk is in order....at least.

Great post by the way, I love when criticism is constructive. Same for the comments so far, I'm impressed, then again thats why Dajngo is such a useful framework.

Re: The Problem With Django

metajack — Wed, 11 Jun 2008 09:50:00 -0000

@James:

Here is what you said in that thread:

"We've stated in the past that the next official supported release of
Django will be 1.0. I think that's a good idea, and I personally think
we ought to stick to it; the things we've outlined for 1.0 are
significant enough that incrementally pushing releases before that
will do more harm (in terms of presenting a constantly-moving target
to the people who believe releases mean long-term stability) than
good. "

You believe that doing intermediate releases will cause more harm than good. I disagree. When has making a release ever done harm? Who will it cause harm to? You personally because of the extra work? Me as a user who is not forced to upgrade to the release (but might)? To new users who just get started and start with code that is over a year old? I do not see how doing more incremental releases causes harm, much less more harm than good.

Also, Django is already a constantly moving target, because most people who are not running big projects are on trunk. Those that are on 0.96 have to maintain private forks to backport patches These are patches that probably would have gotten rolled into interim releases. The longer you wait, the more work it will be for us to make sure that our patches work in newer versions. The longer you wait, the less the code that the big users of Django looks like Django that is in trunk, and the less meaningful feedback you get from those big users.

Re: The Problem With Django

metajack — Wed, 11 Jun 2008 09:42:00 -0000

@James:

I said that ticket was not yet in trunk, and it isn't. It does however have quite a bit of activity. From reading the bug history, there are several places where I think it could have been merged into trunk. After all, it needs a lot of eyes to debug it right? What better place than where all the developer's eyes are? Waiting until the fix is perfect is just as bad as waiting until the release is perfect.

There are numerous other tickets which I also linked which have no clear reason why they aren't yet in trunk, and it has been this way for a year for several of them. The only reaosn I can come up with is because Django is abusing trunk instead of making regular releases. If most of your user base is using trunk, it is far past time for a release.

How has it been practical to avoid releases? Would it not be more practical to do regular releases so that more people are giving you feedback and bug reports? Would it not be more practical to take advantage of the marketing benefits of having regular releases? It's only practical in the sense that it allows you to be lazy and just work on code. I notice that not once have you said _when_ you think 1.0 will be released? My guess is that whatever answer you would give now would be similar to the answer you would have given 6 months or a year ago. Where would GMail be if they waited for 1.0? Early and often releases aren't just good for open source, they tend to be good business in general.

As for delegation, specifically with regards to MSSQL, it sounds to me from my brief reading and listening to others on the subject that the issue is really internal to Django and not with the MS SQL code as it stands. Pagination seems to be the sticking point (and also in Oracle), which could be a clue that Django internals aren't well structured to handle these other databases. If that's so, any external project is doomed to not be very successful unless they are providing a fork of the codebase. Granted, I am no expert on MS SQL, Oracle, or Django pagination, so I am probably way off the mark. But if I were in charge of Django's DB layers, I'd be pretty embarassed and motivated to fix this since Rails has it, and Rails is probably the most well known competition for Django. It seems silly to me that there is clearly a set of programmers who are working on this, and yet the project's main action in this regard is to remove all related code from the tree. Who cares if the code in the tree is unmaintained. Put a note in there about it and start looking for maintainers. This is what most projects do. You've now created a huge barrier to getting this work done in that it's clear that you aren't very interested in MS SQL and have no problems showing casual hackers the door.

You say that there is a public list of things that describe "ready for the next release" and that it's been up for a while. I think it will be up for a while longer (please prove me wrong!). How long has Django been in this mode of "almost ready for release"? Also, "odds are odds are you’d finish QA and upgrades on any deployed projects you have right around the time we’d be ramping up for 1.0 pre-release versions"? I don't think this can be taken seriously. Unless you know how to calculate these odds. Also, assuming it takes a while to do a release, takes a while for places like Chesspark to move to the new code, then here we are, two "a while"s later and you're just starting to "[ramp] up for 1.0 pre-release versions". I'll take that future over the current one. Please.

Re: The Problem With Django

James Bennett — Wed, 11 Jun 2008 09:40:00 -0000

Jack, if you're going to misrepresent what I said, when my actual words are on the same page, I don't know that I can help you much, either.

Re: The Problem With Django

metajack — Wed, 11 Jun 2008 09:20:00 -0000

@Justin: Django is a great framework. It has many fine qualities which is why I use it. I even choose it after have used quite a number of other frameworks (in various languages), and find 0.96 to be a noticeable improvement on them. Like almost everything, it can be improved.

@Peter: I am not a release manager, so my other option would be to fork Django and make my own releases. Instead, I choose to both prod the community to improve the release management and try and educate others doing open source projects so they don't make the same mistakes. In terms of making Django better, we do what we can. We have filed bugs, made patches, done testing, promoted Django, and one of the Chesspark developers is quite active in #django. I have not been shy about promoting Django either.

When users complain about Chesspark (which I assure you they do), we take such complaints seriously and work hard to address their issues. Of course, we have the monetary motivation to make them happy, but at no point do we respond to them "stop complaining".

Re: The Problem With Django

metajack — Wed, 11 Jun 2008 09:10:00 -0000

@pood and @rick:

It sounds like a lot of open source projects, including several of my own, which is probably why it bothers me so much. Releasing early and releasing often is a mantra for a reason, and is something we can only strive to get better at no matter how good we already are.

I will note that a quick look at the Rails milestones shows releases every few months, and they were recently in all the blogs with the latest release they made.

I'll note that each release is a chance to increase mindshare, because releases are topics that people blog about and share widely. How many stories have you read in blogs (outside of the Django planet) that mentioned interesting commits? How many have you read about a release that was made?

The marketing and mindshare potential _alone_ makes releasing often a good idea.

Re: The Problem With Django

metajack — Wed, 11 Jun 2008 09:06:00 -0000

@phillc:

Well they are and they aren't. First let me state that Jacob's suggestion of a timed release schedule is excellent. Neither I, nor Christian, suggested that we tag trunk and throw it up on the site, but I feel that something should be done in the immediate future (say a month?) and should have been done long ago. Nothing like deadlines to get people motivated.

James Bennett however just complains that it takes time and it is hard. Yes, we all know this. That doesn't mean it shouldn't be done. Django already has a release manager, perhaps they can do much of this work so that James and Jacob and the other core developers don't have to; this is delegation. He also says pushing for an interim release will do harm, but I think this is just silly. Did releasing 0.96 cause harm? Can you name a few examples of open source projects that did a release that harmed them? I don't think I could.

As for James' catch-22 argument, maybe the same number of complaints would come in, but there is a crucial difference. Instead of complaining that there are no releases, the complaints would be useful feedback on the releases. Also, there is nothing saying that people have to upgrade "at each step along the way". Upgrading is done by their choice. Would I upgrade Chesspark for every Django release? Of course not. Would I have upgraded from 0.96 to something in the last year? I'm pretty sure I would have. Having releases with release logs gives me a choice and information. Having no releases but some vague plan for 1.0 super release forces me to run my own private fork.

Putting off the problem until 1.0 is not a solution, unless you have hard deadlines for 1.0 and a concrete plan for how to address issues after 1.0. Who's to say 1.0 won't take another year? I certainly don't think that a year ago any of the Django developers thought that 1.0 would take this long. I imagine the conversations were similar then and now, because this is how almost every open source project is. We chase the dragon of perfect software.

Plenty of projects make interim releases that aren't fully supported. The linux kernel, Python, or to name one from my own past, Ogg Vorbis. Also, is support really that big of an issue? Surely those bugs need to be fixed anyway. Wouldn't it be nice to know what they were? Frequent releases help because they get wide spread use. Wide spread use means more eyes on the code and more free QA and but reports. Hording the code in trunk has prevented many from making the code better.

One of the fallacies here is that anyone is in control of Django. It's an open source project. Whether you like it our not, business are already being built atop it. It's not your choice whether it gets used widely. Trying to control the flow of the software is just hurting the project Here's a nice quote from that thread:

"Version 1.0 is going to be the first version that "we"
as a community market to a wider audience. People may have the
opinion that newforms isn't as great as we say it is (and know it is)
if we can't ship our showcase application without it using newforms. "

Django is already being marketed to a wide audience because it has a website and many fans (including myself). What do we see? 0.96 with all its warts and the fact that it is over a year old. Stop caring about what people might think and concentrate on making good software. This is not our first rodeo; we know that software has bugs and gets better over time.

And finally, there is a lot of middle ground between supported, stable release and running off of trunk. There is a lot of false dichotomy in these arguments.

Re: The Problem With Django

Peter Pistorius — Wed, 11 Jun 2008 08:55:00 -0000

"We’ll be here trying to help." By complaining?

Re: The Problem With Django

Justin Hernandez — Wed, 11 Jun 2008 08:50:00 -0000

Great read. Thanks. I still think Django is a great framework, they'll come around eventually.

Re: The Problem With Django

a friend — Wed, 11 Jun 2008 08:48:00 -0000

TG2? one year ago I was told it would have been one in one month. How can I trust them now?

Re: The Problem With Django

James Bennett — Wed, 11 Jun 2008 08:39:00 -0000

Julian, I think you're doing a huge disservice to us all by selectively ignoring things and just repeating an argument over and over when it's been replied to many times already.

There's a very well-defined list of things that describe "ready for the next release". That list is out in public and has been for a while now. And if we rolled a release right this minute, odds are you'd finish QA and upgrades on any deployed projects you have right around the time we'd be ramping up for 1.0 pre-release versions, which would just get you even more upset than you are now.

If you're unwilling to listen, I'm not sure I can do any more to address your concerns.

Re: The Problem With Django

rick — Wed, 11 Jun 2008 08:29:00 -0000

This also sounds like rails development. We've found git to be a big asset in allowing incremental improvements and delegating responsibility to other interested parties.