Opened 15 years ago
Closed 14 years ago
#24310 closed defect (fixed)
python26 does not provide _bsddb
| Reported by: | guy.linton@… | Owned by: | MarcusCalhoun-Lopez (Marcus Calhoun-Lopez) |
|---|---|---|---|
| Priority: | Normal | Milestone: | |
| Component: | ports | Version: | 1.8.2 |
| Keywords: | Cc: | elelay (Eric Le Lay) | |
| Port: | python26 |
Description
I have built python26, but it does not find the _bsddb module:
[Thu Apr 01 10:19:56]G5:~ tim$ sudo python_select python26
Password:
Selecting version "python26" for python
[Thu Apr 01 10:20:36]G5:~ tim$ python
Python 2.6.5 (r265:79063, Mar 21 2010, 12:06:08)
[GCC 4.0.1 (Apple Computer, Inc. build 5367)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import bsddb
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/bsddb/__init__.py", line 64, in <module>
import _bsddb
ImportError: No module named _bsddb
I think that perhaps bdsdb for python26 is in bsddb185.so instead of _bsddb.so. But in my build, it does not seem to be complete. The last bit of otool -TMvDlL for python25 _bsddb.so says:
Load command 3
cmd LC_LOAD_DYLIB
cmdsize 60
name /opt/local/lib/db46/libdb-4.6.dylib (offset 24)
time stamp 1269169817 Sun Mar 21 11:10:17 2010
current version 0.0.0
compatibility version 0.0.0
...skipped
Load command 7
cmd LC_DYSYMTAB
cmdsize 80
ilocalsym 0
nlocalsym 8010
iextdefsym 8010
nextdefsym 5
iundefsym 8015
nundefsym 72
tocoff 0
ntoc 0
modtaboff 0
nmodtab 0
extrefsymoff 0
nextrefsyms 0
indirectsymoff 194972
nindirectsyms 134
extreloff 0
nextrel 0
locreloff 94208
nlocrel 465
/opt/local/lib/db46/libdb-4.6.dylib (compatibility version 0.0.0, current version 0.0.0)
time stamp 1269169817 Sun Mar 21 11:10:17 2010
/usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
time stamp 1159409917 Thu Sep 28 03:18:37 2006
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 88.1.12)
time stamp 1264792999 Fri Jan 29 19:23:19 2010
but for python26, the last part of the otool output is:
Load command 6
cmd LC_DYSYMTAB
cmdsize 80
ilocalsym 0
nlocalsym 1104
iextdefsym 1104
nextdefsym 1
iundefsym 1105
nundefsym 41
tocoff 0
ntoc 0
modtaboff 0
nmodtab 0
extrefsymoff 0
nextrefsyms 0
indirectsymoff 30680
nindirectsyms 75
extreloff 0
nextrel 0
locreloff 16384
nlocrel 68
/usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
time stamp 1159409917 Thu Sep 28 03:18:37 2006
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 88.1.12)
time stamp 1264792999 Fri Jan 29 19:23:19 2010
Please let me know what flags for otool are needed if you want the rest of the otool output.
Attachments (1)
Change History (22)
comment:1 Changed 15 years ago by mf2k (Frank Schima)
| Cc: | blb@… removed |
|---|---|
| Keywords: | bsddb removed |
| Owner: | changed from macports-tickets@… to blb@… |
comment:3 Changed 15 years ago by blb@…
How did you build db46, python25, and python26?
$ port installed db46 python25 python26
Usually when bsddb (or a few other modules) have issues, there's a problem linking to the dependents (then, unfortunately, python continues to build and simply doesn't install a working module).
comment:4 follow-up: 5 Changed 15 years ago by guy.linton@…
I suppose you mean port install.. not installed. I just did
sudo port build <pointing at a local copy of gramps Portfile>
This builds in the order db46 python26 python25. Could the problem be that I have built python26 before python25? Should python26 be dependent on python25, or is there some other dependency missing from python26? Please note that my build was starting from a completely fresh installation of MacPorts with nothing built previously. Can you tell me what you get for otool -TMvDlL bsddb185.so.
List of build order:
---> Building expat ---> Building gperf ---> Building libiconv ---> Building ncursesw ---> Building ncurses ---> Building gettext ---> Building lzmautils ---> Building texinfo ---> Building aspell ---> Building aspell-dict-en ---> Building xmlcatmgr ---> Building docbook-xml-4.1.2 ---> Building docbook-xml-4.2 ---> Building docbook-xml-4.3 ---> Building docbook-xml-4.4 ---> Building docbook-xml-4.5 ---> Building docbook-xml-5.0 ---> Building docbook-xml ---> Building docbook-xsl ---> Building iso-codes ---> Building zlib ---> Building libxml2 ---> Building libxslt ---> Building perl5.8 ---> Building perl5 ---> Building p5-xml-parser ---> Building pkgconfig ---> Building bzip2 ---> Building db46 ---> Building gdbm ---> Building openssl ---> Building readline ---> Building sqlite3 ---> Building freetype ---> Building fontconfig ---> Building xorg-bigreqsproto ---> Building xorg-inputproto ---> Building xorg-kbproto ---> Building xorg-xproto ---> Building xorg-libXau ---> Building xorg-libXdmcp ---> Building xorg-util-macros ---> Building xorg-xcmiscproto ---> Building xorg-xextproto ---> Building xorg-xf86bigfontproto ---> Building xorg-xtrans ---> Building xorg-libX11 ---> Building xorg-renderproto ---> Building xrender ---> Building Xft2 ---> Building tcl ---> Building xorg-libXext ---> Building xorg-scrnsaverproto ---> Building xorg-libXScrnSaver ---> Building tk ---> Building python26 ---> Building py26-libxml2 ---> Building getopt ---> Building gnome-common ---> Building p5-getopt-long ---> Building p5-pathtools ---> Building p5-scalar-list-utils ---> Building intltool ---> Building rarian ---> Building gnome-doc-utils ---> Building gtk-doc ---> Building p5-locale-gettext ---> Building help2man ---> Building m4 ---> Building autoconf ---> Building automake ---> Building libtool ---> Building glib2 ---> Building atk ---> Building libpixman ---> Building libpng ---> Building cairo ---> Building jpeg ---> Building jasper ---> Building pango ---> Building shared-mime-info ---> Building tiff ---> Building xorg-compositeproto ---> Building xorg-fixesproto ---> Building xorg-libXfixes ---> Building xorg-libXcomposite ---> Building xorg-libXcursor ---> Building xorg-damageproto ---> Building xorg-libXdamage ---> Building xorg-libXi ---> Building xorg-xineramaproto ---> Building xorg-libXinerama ---> Building xorg-randrproto ---> Building xorg-libXrandr ---> Building gtk2 ---> Building libglade2 ---> Building gdl ---> Building db47 ---> Building flex ---> Building dbus ---> Building dbus-glib ---> Building popt ---> Building desktop-file-utils ---> Building libidl ---> Building orbit2 ---> Building policykit ---> Building gconf ---> Building gnome-mime-data ---> Building gnome-vfs ---> Building libgpg-error ---> Building libgcrypt ---> Building libtasn1 ---> Building gnome-keyring ---> Building lzo ---> Building opencdk ---> Building gnutls ---> Building xorg-libice ---> Building xorg-libsm ---> Building xorg-libXt ---> Building xorg-libXmu ---> Building libproxy ---> Building libsoup ---> Building libgda4 ---> Building libgtkhtml ---> Building libbonobo ---> Building audiofile ---> Building esound ---> Building libgnome ---> Building libart_lgpl ---> Building libgnomecanvas ---> Building libbonoboui ---> Building hicolor-icon-theme ---> Building p5-xml-namespacesupport ---> Building p5-xml-sax ---> Building p5-xml-simple ---> Building icon-naming-utils ---> Building gnome-icon-theme ---> Building libgnomeui ---> Building gmp ---> Building mpfr ---> Building gcc43 ---> Building gzip ---> Building atlas ---> Building fftw-3 ---> Building python25 ---> Building py25-setuptools ---> Building py25-nose ---> Building py25-numpy ---> Building py25-cairo ---> Building py25-gobject ---> Building py25-gtk ---> Building py25-orbit ---> Building py25-gnome ---> Building gnome-python-extras ---> Building hunspell ---> Building enchant ---> Building gtkspell2 ---> Building py25-bsddb ---> Building py25-hashlib ---> Building py25-zlib ---> Building gramps ---> Building python_select
comment:5 Changed 15 years ago by blb@…
Replying to guy.linton@…:
I suppose you mean port install.. not installed. I just did
I actually did mean installed; when you run it, you'll see (it shows how those ports are installed).
comment:6 Changed 15 years ago by guy.linton@…
Sorry, The answer is:
The following ports are currently installed: db46 @4.6.21_6 (active) python25 @2.5.5_0+darwin+darwin_8+macosx (active) python26 @2.6.5_0+darwin (active)
comment:7 Changed 15 years ago by blb@…
Okay, no universal stuff going on. What is your Xcode version, as older gcc versions shipped with older Xcodes have caused issues? 2.5 is latest/last for 10.4 (see the FAQ for info on upgrading if you have an older one).
comment:8 Changed 15 years ago by guy.linton@…
Yes, just upgraded because some component demanded a newer version - I have Xcode 2.5.
comment:9 Changed 15 years ago by blb@…
Was that after you installed the python26 port (check the date of /opt/local/var/macports/receipts/python26/2.6.5_0+darwin/receipt.bz2 to see when it was installed). If the Xcode upgrade happened after that, then definitely try uninstalling/reinstalling the python26 port.
If, however, if was before then I take it the file /opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-dynload/_bsddb.so does not exist? If it doesn't, what is the output from
$ grep DB_VERSION_STRING /opt/local/include/db46/db.h
comment:10 Changed 15 years ago by guy.linton@…
The upgrade to Xcode was AFTER I installed the python26 port.
python26 was installed on 21 March, but the upgrade to Xcode was 22 March when I had "libtheora 1.1.1 requires Xcode 2.5 or later but you have Xcode 2.4.1." (Xcode modification date is 22 Mar 14:01, while the python26 receipt is 21 Mar 12:11).
I will try uninstalling/reinstalling python26.
[The file /opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-dynload/_bsddb.so does NOT exist. The output from the grep is #define DB_VERSION_STRING "Berkeley DB 4.6.21: (September 27, 2007)"]
comment:11 Changed 15 years ago by guy.linton@…
I have now done
sudo port -f uninstall python26 sudo port install python26 sudo python_select python26 python /opt/local/share/gramps/gramps.py
Gramps starts OK, but when it tries to load the most recent database, it complains about the format of the Berkley database (Database needs recovery, cannot open it!
Low level database corruption detected Gramps has detected a problem in the underlying Berkeley database. This can be repaired by from the Family Tree Manager. Select the database and click on the Repair button DB_RUNRECOVERY: Fatal error, run database recovery -- process-private: unable to find environment
After supposedly repairing the database, when I then try to open it I get an"Unexpected Error":
86421: ERROR: gramps.py: line 138: Unhandled exception
Traceback (most recent call last):
File "/opt/local/share/gramps/gui/dbman.py", line 649, in __repair_db
restore(dbase)
File "/opt/local/share/gramps/gen/db/backup.py", line 153, in restore
__do_restore(database)
File "/opt/local/share/gramps/gen/db/backup.py", line 170, in __do_restore
database.rebuild_secondary()
File "/opt/local/share/gramps/gen/db/write.py", line 222, in try_
raise Errors.DbError(msg)
DbError: Invalid argument -- /Users/tim/.gramps/grampsdb/4b229499/person_id.db: unexpected file type or format
So, it seems that although the database is found OK, there is something seriously wrong with the database build as found by python 2.6 (or at least it is not compatible with that from python 2.5).
comment:12 follow-up: 13 Changed 15 years ago by guy.linton@…
I should have said that after doing the reinstall, /opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-dynload/_bsddb.so DOES exist.
comment:13 Changed 15 years ago by blb@…
Replying to guy.linton@…:
I should have said that after doing the reinstall, /opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-dynload/_bsddb.so DOES exist.
Okay, that suggests that Xcode 2.4.1 had some problem with building the bsddb module that 2.5 fixed; I guess we can call that the good news.
As far as the database incompatibilities, that's definitely weird; python25 uses the same version of BDB (db46) as python26 so you'd think they would be compatible.
There is a gramps bug that reads similar to your issue. Perhaps the steps there (using db46_recover which should be installed with the db46 port) may help?
comment:14 Changed 15 years ago by guy.linton@…
As you say, the bsddb module now seems to be available to python26.
As you say, the problem is weird. The failure is reported here:
try:
try:
self.dbstate.db.load(filename, self._pulse_progress,
mode, upgrade=False)
except gen.db.exceptions.DbUpgradeRequiredError, msg:
if QuestionDialog2(_("Need to upgrade database!"),
str(msg),
_("Upgrade now"),
_("Cancel")).run():
self.dbstate.db.load(filename, self._pulse_progress,
mode, upgrade=True)
self.dbstate.db.set_save_path(filename)
else:
self.dbstate.no_database()
except gen.db.exceptions.DbVersionError, msg:
self.dbstate.no_database()
self._errordialog( _("Cannot open database"), str(msg))
except OSError, msg:
self.dbstate.no_database()
self._errordialog(
_("Could not open file: %s") % filename, str(msg))
except Errors.DbError, msg:
self.dbstate.no_database()
self._dberrordialog(msg)
except Exception:
self.dbstate.no_database()
Where _dberrordialog is being called. The msg attached is:
DB_RUNRECOVERY: Fatal error, run database recovery -- process-private: unable to find environment
It doesn't seem to be that the database is corrupt, but rather that something is wrong with the 'environment'. Do you have any ideas what dbload may be trying to say. What is this environment it is trying to find?
db46_recover doesn't seem to make any difference.
There are known incompatibilities between the databases in Gramps 3.1 and Gramps 3.2. These are due to a mistake in Gramps about block size (or something like that). The work-around is to export the database from 3.1 (in a Gramps format) and then import into 3.2. The problem has now been fixed in a forthcoming release of Gramps.
So, I have tried importing the Gramps dump into a fresh python26 database, and that works OK. Switching back to python25 gives a message about database corruption (not the same environment message).
However, when I run Gramps with python26, I get the slow running reported in http://www.gramps-project.org/bugs/view.php?id=3750. I don't think the switch to the more recent database is possible because there is no Macports version of it.
There is clearly a difference between the behaviour of
5316156 520 -rwxr-xr-x 2 root wheel 263404 Mar 21 22:12 ./local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/lib-dynload/_bsddb.so 6113516 640 -rwxr-xr-x 2 root wheel 325416 Apr 8 22:44 ./local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-dynload/_bsddb.so
In conclusion (1) Any idea how I could find out what the problem with switching between the two different implementations of _bsddb.so is? (At least in one direction, it seems to be that the location of the environment is lost). (2) The python 2.6 version of the database is slow.
comment:15 follow-up: 16 Changed 15 years ago by blb@…
Sounds as if the issue may be with python 2.6's bsddb, at least reading over the gramps bug, right? Has anyone filed a bug with python? I didn't see anything with a quick search; could be helpful.
The MacPorts build shouldn't be doing anything weird with that module (you can see all the patches which go into the port in trac as well as the Portfile itself. Nothing there is changing bsddb-specific functionality or patching, as far as I can tell.
comment:16 Changed 15 years ago by guy.linton@…
Replying to blb@…:
Sounds as if the issue may be with python 2.6's bsddb, at least reading over the gramps bug, right? Has anyone filed a bug with python?
I have raised a bug with python "issue8504 bsddb databases in python 2.6 are not compatible with python 2.5 and are slow in python 2.6" for this.
comment:17 Changed 14 years ago by blb@…
| Cc: | mcalhoun@… removed |
|---|---|
| Owner: | changed from blb@… to mcalhoun@… |
comment:18 Changed 14 years ago by elelay (Eric Le Lay)
hi, jumping into this because I would like to test gramp...
I can't have the _bsddb.so module built with python26 @2.6.6_0+no_tkinter. I'm on 10.4.11 ppc G4 XCode 2.5 and uninstalled db46 python26 and then reinstalled this afternoon, with no success.
running sudo port -d destroot python26, this is what I found :
Failed to find the necessary bits to build these modules: _bsddb linuxaudiodev ossaudiodev spwd sunaudiodev To find the necessary bits, look in setup.py in detect_modules() for the module's name.
So bsddb46 is not found. To my mind, it is due to this change, because in the list of working architectures there isn't "Power Macintosh", so setup.py ignores the installed db46. The change dates back to april, so it wasn't there in the version of python Guy tried. Overwise I don't understand how he managed to build _bsddb.so
Indeed, when I edit setup.py line 805 to add 'Power Macintosh' like this :
if arch not in ('i386', 'i486', 'i586', 'i686',
'x86_64', 'ia64', 'Power Macintosh'):
_bsddb.so is built. I'll test some more and provide a patch.
comment:20 Changed 14 years ago by elelay (Eric Le Lay)
with this patch, _bsdb is built and working.
comment:21 Changed 14 years ago by elelay (Eric Le Lay)
| Resolution: | → fixed |
|---|---|
| Status: | new → closed |
the upstream bug has been closed as invalid. applying the patch in r72668. To sum-up, the issue in python26@2.6.6_0 is that python rejects db46 on ppc because it's allegedly unstable, so the bsddb module is not built.
FYI, it works fine for me. I'm on Intel Snow Leopard. What OS version and architecture are you running? It looks like you are on 10.5 right?