Building PyLucene for Review Board on Ubuntu Hardy


My team (well, there’s just two of us now) is creating the development infrastructure for our big project, which is building my company’s new CMS on Plone. We’ll be Agile, of course, but without zealotry.

It fell to me to install Review Board on our Ubuntu 8.04 (Hardy) administrative server, to evaluate it for our online code reviews. This should have been a snap, because:

  1. The server is a clean installation, administered by yours truly. I install only necessary software, and use apt-get for all of them.
  2. I read installation instructions before installing software.
  3. I’m smart.

I installed Review Board and hooked it up to Apache 2.2 without too much trouble. (I had a mental block on the <Alias> directive, but never mind that.)

All went well, until I got to the end of Review Board’s installation instructions. There I found:

Set Up Search
This isn’t strictly necessary (and might be hard on some platforms), but it’ll make your users love you. See SettingUpSearch for details.

I want my team to love me. So, I took the bait.

ReviewBoard search uses PyLucene. There’s no PyLucene package for Hardy. This should have told me something. But it had build instructions, so it should be no big deal. Right? I plunged ahead.

apt-get install gcc g++ sun-java6-jdk ant worked fine. Looking good.

I went for the final install, and it blew up.

root:~/PyLucene-2.4.0-2/jcc# python setup.py build
running build
running build_py
copying jcc/config.py -> build/lib.linux-x86_64-2.5/jcc
copying jcc/classes/org/osafoundation/jcc/PythonVM.class -> build/lib.linux-x86_64-2.5/jcc/classes/org/osafoundation/jcc
copying jcc/classes/org/osafoundation/jcc/PythonException.class -> build/lib.linux-x86_64-2.5/jcc/classes/org/osafoundation/jcc
running build_ext
building ‘jcc’ extension
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -D_jcc_lib -I/usr/lib/jvm/java-6-openjdk/include -I/usr/lib/jvm/java-6-openjdk/include/linux -I_jcc -Ijcc/sources -I/usr/include/python2.5 -c jcc/sources/jcc.cpp -o build/temp.linux-x86_64-2.5/jcc/sources/jcc.o -DPYTHON -fno-strict-aliasing -Wno-write-strings
cc1plus: warning: command line option “-Wstrict-prototypes” is valid for Ada/C/ObjC but not for C++
jcc/sources/jcc.cpp:20:17: error: jni.h: No such file or directory
In file included from jcc/sources/JObject.h:21,
                 from jcc/sources/jcc.cpp:28:
jcc/sources/JCCEnv.h:73: error: ‘jobject’ does not name a type
jcc/sources/JCCEnv.h:79: error: ‘jclass’ does not name a type
[…snip…]

Ruh roh, jni.h was nowhere to be found!

After a bit of sleuthing, here’s what the problem turned out to be. The installation instructions — which had been tested and verified to be correct, right? guess not — had me install the sun-java6-jdk package, which created /usr/lib/jvm/java-6-sun-1.6.0.07 and an associated /usr/lib/jvm/java-6-sun symlink.

But, gee whiz, the gcc command looked for includes in /usr/lib/jvm/java-6-openjdk.

The Sun JDK and Open JDK are supposed to now be functionally identical, with the latter one just having a more friendly open-source license. So, I edited setup.py and changed the INCLUDES to be:

INCLUDES = {
[snip]
'linux2': ['/usr/lib/jvm/java-6-sun/include', '/usr/lib/jvm/java-6-sun/include/linux'],
[snip]

Rebuild.

Bam, crash and burn with a g++ error.

g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions build/temp.linux-x86_64-2.5/jcc/sources/jcc.o build/temp.linux-x86_64-2.5/jcc/sources/JCCEnv.o -o build/lib.linux-x86_64-2.5/libjcc.so -L/usr/lib/jvm/java-6-openjdk/jre/lib/amd64 -ljava -L/usr/lib/jvm/java-6-openjdk/jre/lib/amd64/server -ljvm -Wl,-rpath=/usr/lib/jvm/java-6-openjdk/jre/lib/amd64:/usr/lib/jvm/java-6-openjdk/jre/lib/amd64/server -Wl,-S -lpython2.5
/usr/bin/ld: cannot find -ljava
collect2: ld returned 1 exit status
error: command 'g++' failed with exit status 1

Ah, the pesky “openjdk” again. Turns out the link flags were also broken. I changed the LFLAGS to also point to /usr/lib/jvm/java-6-sun.

Rebuild.

Success! Finishing the installation required some fiddling with my sys.path, but that was trivial.

8 comments
  1. Pasi Kuusisto said:

    First, thanks for the instructions! I hit the same barrier.

    Is there any reason you just didn’t symlink the java-6-openjdk to point to java-6-sun? I did it that way and it seemed to do the trick just fine compilation-wise. Still have a ways to go before the thing is hooked up to Apache and running.

    -zi

  2. John said:

    @Pasi: I prefer to not symlink between different pedigrees. If I do it once or twice, it’s no big deal. But the phrase, “I’ll do it this just once,” is one of those “famous last words” that comes back to bite me.

    So, if I have /foobar-v2 and /foobar-v1, I have no issue with symlinking /foobar to /foobar-v2. But symlinking /foobar to /fred-v4 is a line I don’t want to cross.

    TEHO. TMTOWTDI. 🙂

  3. Pasi Kuusisto said:

    John,

    I completely agree (also with your acronyms), and I only intended to leave the symlink there until the thing is compiled (yeah, right).

    Being pretty much a Linux n00b (I did use the school’s unices way back in the PDPzoic era, but even then only as a (l)user), so I’m learning as I go. That’s really why I asked, to learn if maybe I was setting myself up for a spectacular disaster.

    Our box was still running Gutsy, and I ran into more trouble even after a successful compilation. Upgraded to Hardy, then left it churning the Intrepid upgrade for tonight. I’ll get back to fiddling with it in the morning. Planning to push in Jaunty Real Soon Now ™ (not the beta just yet, though).

    Thanks for the reply. I’ll be sure to check back, probably among the first things in the morning, I have a hunch this struggle is still far from over.

    Br,

    -zi

  4. Wow thanks! Your steps made mine work, someone should inform them to change it to make it work.. I had to rename them all to “java-1.6.0-openjdk”

  5. Thanks so much for this documentation. Unfortunately I still need help. Any chance you could post what your LFLAGS finally looks like? I am a newbie to linux build from src and I have no idea what setup.py should finally be.

    Thanks in advance.

    • John said:

      @Ganeshram: Since this post, I’m no longer employed by Fisher Communications. So I don’t have access to that setup.py. And my entire team was laid off, so there’s nobody left for me to ask on the side.

      IIRC, for LFLAGS I replaced every instance of “/jvm/java-6-openjdk” with “/jvm/java-6-sun”. I then rebuilt, and it worked.

      I’m surprised this hasn’t been fixed in the six months since I encountered it!

  6. ramesh said:

    I got same error that -lpython2.5 not found on ubuntu. But your trick is not working in my case.
    can you give me some idea to fix this.

    Thanks
    Ramesh

  7. Mamta said:

    Thanks a ton.. That helped me resolve my pylucene issues after a crazy day of head scratching

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: