urllib2 gotcha

Does this return all the data?

urllib2.urlopen(url).read()

The answer appears to be "not always".

http://stackoverflow.com/questions/7174927/when-does-socket-recvrecv-size-return https://github.com/CVL-dev/feedparser/issues/1

I can't seem to get a definitive answer here, nor a reproducible demonstration of it failing.

Does anybody know for …

more ...


My Emacs setup

Some notes on my emacs setup. Maybe this should go in README for my dotemacs.

Python

Which mode?

There is python-mode.el, and python.el. I have settled on python-mode from the elpa repositories.

Rope?

I tried ropemacs, and really like it when it works. For simple refactorings it can …

more ...


convert hg repository to git

How to create a brand new git repository from an existing hg repository. This is not explained on http://hg-git.github.com/

  easy_install hg-git
  # Now edit your ~/.hgrc as per http://hg-git.github.com/

  # Now make a bare git repository. Yes, bare.
  git init --bare ~/my-git-repo    # the --bare is important …
more ...

Linux on the Macbook Air (2013)

For various reasons that might be explained in more retroblogging, around January 2014 I got fed up with OSX and decided to switch back to Linux. I got things more or less working on my Macbook Air. (Later, in 2016, replaced with a Macbook Pro.)

TL;DR: It's okay, and …

more ...

settings.py considered evil

Django's settings.py is evil.

Why? Because in terms of security, it is a bomb waiting to go off. Typically your database settings go in there, and potentially other sensitive stuff (salts, service API keys, what have you).

By encouraging you to put a settings.py file in your project's …

more ...

Python string split weirdness

Consider:

>>> ''.split()
[]

That seems okay... split an empty string on whitespace and you get an empty list, sure, why not.

Except that in every other case, if you split a string on a separator that it doesn't contain, you get a list containing the original string - even if the original …

more ...

Context Manager + Decorator

(Updated 2018 with a more modern Python 3 tip)

I always forget how to write a class that can be used as both a decorator and a context manager. It's a handy trick, but I don't do it often enough to remember, so, here it is.

In Python 2, you …

more ...

map vs. iter

What's the difference between these?

>>> zipped = zip(iter1, iter2)
>>> mapped = map(None, iter1, iter2)

The answer turns out to be depend on whether iter1 and iter2 are the same length. Pretty subtle: zipped is truncated to the length of the shortest of (iter1, iter2). mapped is the same length as …

more ...


Debugging as Immersive Game

Debugging with print statements is kind of like the old 2D Mario games. You get a little bit forward, learn a little more about the world you're exploring until something sends you back to the beginning. You get a little bit farther each time, but you keep having to go …

more ...

Django is not an application framework

Django as it is in the wild is not a web software framework. It is a web site framework.

(disclaimer re. complaining: http://commandcenter.blogspot.com/2011/12/esmereldas-imagination.html )

What does this mean and why?

For some things, I love Django. It hits a particular sweet spot really well …

more ...

Mock configuration tip

A little tip for working with the mock library. Let's say you need to set up some child attributes, like for example you're mocking up a django.contrib.auth.User, and you want it to be a logged in user:

>>> user = mock.Mock()
>>> user.username = 'Bob'
>>> user.is_authenticated.return_value = True …
more ...

Python packaging: Where Pip and Virtualenv fall short

I love the combination of virtualenv and pip requirements files. This is a great 95% solution to the problem of deploying and distributing known-good sets of python packages, safely (semi-)isolated from other python environments on the same system.

The --no-site-packages option is great because it makes your installation isolated …

more ...

Cron rant

The venerable Cron in its various forms (vixie-cron, anacron, etc) is good at what it does, but there are a lot of things that could make it much better to work with. For example:

  • "Debug" mode: run all the rules in a given crontab file right now and put errors …

more ...

Python dictionary trick: d3 = d1 + d2

Want to merge two dictionaries into a new third dictionary without modifying either of the originals? Easy:

>>> dict3 = dict(dict1, **dict2)

Keys in dict2 would override matching keys in dict1. And, as usual, mutable values would be preserved by reference...

Personally I've always wished that dicts would define __add__ to …

more ...

Migrating Trac Wikis to Github

I haven't automated this yet, but will if I ever do this again.

  • Get a trac wiki export: trac-admin $PWD wiki dump /tmp/wiki-dump/

  • cd /tmp/wiki-dump

  • Rename the home page: mv WikiStart Home

  • Clean up trac-specific junk: rm -f Wiki* Trac* CamelCase BadIP PageTemplates Inter* SandBox TitleIndex RecentChanges

  • Trac …

more ...


Reduce Friction for Your Bug Reporters

If you want your users to report bugs, don't put hurdles in their way.

Today I was trying to convert a word processing document to plain text and make it look reasonably nice. I started with OpenOffice's "save as plain text" feature, but it wasn't handling tables very well.

So …

more ...