readline, Python, IPython, and Mac OS X

June 11, 2009 at 8:18 pm
filed under Technology
Tagged , , ,

All right, so maybe I’ll finally use this space.

If you’ve tried to get IPython working on Mac OS X Leopard (that’d be 10.5, and 10.5.7 in particular) with an Intel based CPU, you’ve probably had some problems. I know I did! Let me cut to the chase, with a more technical explanation to follow. Oh, and some mild bitching, too.

I’m following the instructions I originally saw here, on IPython’s Launchpad site.  Yes, my instructions are merely a reprint of someone else’s bug report. More on that after I explain.

Quick instructions

I’m going to assume you haven’t downloaded anything, although if, like me, you wrestled with this for a while, you can still do these steps and it’ll work. It worked for me, anyway. :/

  1. Download the .egg for readline, presumably the latest version. Get IPython if you haven’t already. As of this writing, you can get it here at the IPython download site. You can s/i386/fat/ if you like.
  2. Copy it to a directory, such as ~/python.
  3. You can try to install readline and IPython: sudo easy_install readline ipython. One or both will fail.
  4. Open /usr/local/bin/ipython in your favorite text editor, and replace the contents of the file as described at the IPython Launchpad bug linked above. The key is to hardcode readline into the Python system path immediately after sys, and then to launch IPython explicitly.
  5. Launch ipython.

You should be OK now, tabbing and all. If you haven’t checked this shit out, using IPython for your system shell, I highly recommend that you do. It’s none too shabby. There’s a book on this, too.

Technical stuff after the jump.

wtf, man

So, here’s the skinny. It took me an hour or two to figure this out, which is way too long. I’m writing this post so that hopefully I can nail some of the search terms that weren’t around when I looked for it on the Googles. I’m mentioning a whole bunch of keywords and specific stuff in the hopes that the next person who has this problem gets here or to the Launchpad site and manages to solve their problem.

Anyway, the technical background: IPython wants GNU’s readline library (tabbing!), but Mac OS X ships with libedit. If you somehow manage to get IPython to work, tabbing won’t because IPython is using libedit. It sucks and practically defeats the purpose of IPython. I had this working briefly, and then I broke it somehow. Whee!

You can download the readline egg and try to install it with easy_install. It will try to compile, then error out. In particular, it errors out on Modules/readline.c:681 a bunch of times. I’m not 100% sure why it does this, and I’m too lazy to remember/figure it out again. I think it’s trying to compile the readline lib specifically for Python and failing because of some mismatch. It sucks and it makes me sad inside.

When I tried to run IPython anyway, I got a bunch of errors about unable to find an entry point. I suspect this is some failure related to  easy_install, egg, or pkg_resources and readline being missing, although it’s vastly unintuitive if so. You’ll get an ImportError related to console_scripts and ipython. It’ll say entry point not found.

The fix appears to be skipping the egg, easy_install, pkg_resource shenanigans by explicitly adding readline to your path and then explicitly invoking IPython instead of giving something else a chance to error out. Sounds good to me!

readline + IPython + Fink?

I didn’t try doing anything with Fink. Fink does have both a version of IPython and readline, so it’s possible that Fink would be a better choice overall. Honestly, I’d try that before anything else. I was too stubborn to give up halfway through.

Of course, not having tried it, I don’t know that it’ll work. It’s gotta be less painful than the hackery above, though, right? Right? I mean, what could possibly go wrong?

  • Pingback: Getting IPython readline and auto-completion to work on Mac OS X | Patrick's playground()

  • denis

    Does ipython help work for you in a Terminal, if so what’s TERM please ?
    Here with TERM xterm-color (don’t know who sets that)
    anycommand? gives junk like
    ESC[1;31mType: ESC[0mfunction
    ESC[1;31mBase Class: ESC[0m

    (Does ipython use libcurses or what ?)

    My versions: mac 10.4.11, py 2.5.1, ipython 0.9.1, Terminal v1.5, PAGER less v 418
    Thanks, cheers
    — denis

  • Matthew

    Hrm. I am not sure about this, honestly. :( My TERM is set to xterm-color. You can change that in Terminal preferences.

    I do know that IPython for Linux would use readline (maybe libcurses; can’t speak to that), but my recollection is that it either cannot use readline on OS X or there’s bad behavior. As for that, I’m sure you’ve guessed that those are color codes. I’m not sure why the Terminal wouldn’t interpret those as color, though; the only time I have that problem is when I manage to pipe colorized output to some utility that doesn’t like it, typically when I’ve set –color=always (or the equivalent) instead of –color=auto (or the equivalent).

    Is there a reason why you mentioned your pager? :)

  • Tim

    I had this problem as well, for me it only occured with iPython when showing help on Numpy functions. I would get something like:

    In [1]: import numpy as N

    In [2]: N.sort?

    ESC[1;31mType: ESC[0mfunction
    ESC[1;31mBase Class: ESC[0m
    ESC[1;31mString Form:ESC[0m

    while colour on the iPython prompt itself *did* work.

    The hint about the PAGER got me thinking and it appears that using ls -R as PAGER solves the problem. Using

    export PAGER=”less -R”

    and restarting iPython does the trick. According to the ls man page, -R does not parse the ANSI color codes:

    -R or –RAW-CONTROL-CHARS
    Like -r, but only ANSI “color” escape sequences are output in
    “raw” form. Unlike -r, the screen appearance is maintained cor-
    rectly in most cases. […]

    I don’t know why this works exactly, but it works for me. HTH for people googling on this.

%d bloggers like this: