screeley.com

A Faster Python Script for Extracting Excerpts from Articles

A couple weeks ago David Ziegler posted an article on how to extract excerpts from articles using Python and BeautifulSoup. It works well, but I would like to suggest some improvements by using lxml instead. It's a fairly simple problem. Get the title and the description out of the head, and if there is no description, try to pull some content out of the body. First two easy and the last one sucks, but Python has tools that make our life easier. BeautifulSoup is the go to for web scraping in Python, but it suffers when it comes to performance. lxml is definitely faster and in this case about 3 times so.

• at 4:22p.m.
• permalink
• 4 comments

Django Alfresco

I have a wheelhouse and it's integrating Django with a Java Open Source project. Today I get to announce the next one, Django Alfresco. We combined the Alfresco's document management capabilities with Django's web tier components. I get mixed reactions when I tell people about this project. Anywhere from, "Why did you go and mess up a good thing" to "This is amazing." The former more than the latter, but I'm going to try to convince you that it is a really good idea to use this project. Jeff Potts who is the ECM lead at Optaros and got the project to a place where it could be released has a post on it and a screencast.

• at 4:22p.m.
• permalink
• 4 comments

Django Daemon Command Extension

Recently I started to move Cubby Scott away from a cron and towards a queue. It's hard to be real time when you wake up a cron job once every 3 minutes. Lame. I'm also in the process of adding screenshots and content retrieval. Both take a good amount of time to process. The queue part was easy after reading Rabbits and Warrens and Working with Python and RabbitMQ. The problem came when I started working on the consumer, no one ever talks about the consumer. Well I'm going to give the consumer some love.

• at 4:22p.m.
• permalink
• 4 comments

CubbyScott.com | An experiment in 140 character requirements

Is there a 3rd party twitter app that builds a link page based on my follows? If not, someone should build it. It would be my start page.

Fred Wilson posted this tweet a few days ago, a pretty simple requirement. Get all users that Fred is following, parse, get the links and display them for Fred's viewing pleasure. Personally I really like this idea. The problem with an asymmetrical relationship is that you really only follow that person for the interesting links they post. I follow mostly tech people and honestly, their personal comments don't really do much for me. It would be great if I could get all those links into one feed and filter out all the noise.

So in the last 4 days I put together an application to do this. Personal web developer to Fred Wilson and hopefully a few others out there.

• at 4:22p.m.
• permalink
• 4 comments

Class Based Views and Reusable Apps

One of the major pain points of using reusable apps is modifying the view logic. No matter how many options you can pass into the view function, someone is going to want to do something different with it. I've been using Pinax for a little while now and modifying views is the only thing I don't love about it. Enter class based views. There is currently a ticket out there to make them part of Django 1.2 and there is a great example out on djangosnippets.org. Instead of trying to deal with everything in the url conf you get a class object with functions that you can override. The simple user will not know the difference between the two, while the advanced user can create custom views easily, without repeating logic. Who likes DRY?

• at 4:22p.m.
• permalink
• 4 comments

Denormalization and Managers

This is a "This is how I solved a problem, I know there is a better way, so someone please tell me" post. Denormalization is something that has gotten a bit of press in the Django community. David Cramer has a great post on a model to handle this and there was some discussion on adding a DenormalisationField to have this to Django core. I ended up needing to use a denormalized field on a project, but still wanted to use managers to handle the related field.

• at 4:22p.m.
• permalink
• 4 comments

Code Reviews with Review Board

I went through the process of installing Review Board last night and was happily surprised and deeply angered at the same time. I'm pleased to report that most my anger was taken out on CentOS. Review Board is a Django powered tool for code reviews developed by VMware and subsequently Open Sourced. If you ever have some free time, browse around their repository, it's pretty impressive. I'll talk about my experience of installing it, but mostly I'll stick to the processes behind the code reviews.

• at 4:22p.m.
• permalink
• 4 comments

Django Solr

After writing about a thousand lines of documentation, I'm spent. I'd like to write something witty or clever or fun, but I got nothing.

With some Optaros blessing I released a Django Solr module out into the wild. It was written by Jay Dolan for a top 20 newspaper site about 9 months ago, but had to be rewritten for Django 1.0 and I got side tracked with clients and Alfresco.

• at 4:22p.m.
• permalink
• 4 comments

Django and robots.txt

In the spirit of SEO, Django and this site I took the next step and added a robots.txt file. Nothing too exciting here, but I saw some postings on how to do it the wrong way that made me sad. Like serving the robots.txt file via Django. Why are you adding the overhead of django/mod_python to serve a static file?

• at 4:22p.m.
• permalink
• 4 comments

Django Sitemaps and a Better ping_google Function

In an effort to play around with the sitemaps contrib package I decided to add a sitemap to this website. It's pretty intuitive and easy, so there is really no excuse not to put a sitemap on your site. It's about 15 minutes worth of effort and 2 changes. The effort comes when you don't want to ignore every search engine other than Google. Let's start by adding a sitemap then we'll show how to ping Yahoo and Ask as well.

• at 4:22p.m.
• permalink
• 4 comments

Enterprise Django: Django on Jython

Enterprises like Java. It's safe. No one has ever been fired for suggesting Java. More and more I think the industry as a whole is realizing that Java doesn't work for rapid development. (Groovy on Grails excluded)

So how do you convince a (insert large Java shop here) to take a flyer on one of them there newfangled web frameworks? Well tell them we can run it in a JVM of course.

• at 4:22p.m.
• permalink
• 4 comments

Use Django to assemble instead of Drupal.

First off a little comparison.

Tag lines-

Drupal: "Drupal is a free software package that allows an individual or a community of users to easily publish, manage and organize a wide variety of content on a website."

Django: "Django is a high-level Python Web framework that encourages rapid development and clean, pragmatic design."

Clean and Rapid > Easy: Django +1

• at 4:22p.m.
• permalink
• 4 comments