Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 763609 - dev-db/postgresql-13.1 Migration as documented needs update and doesn't work if PostGIS is involved
Summary: dev-db/postgresql-13.1 Migration as documented needs update and doesn't work ...
Status: RESOLVED CANTFIX
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal major (vote)
Assignee: PgSQL Bugs
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-01-04 23:26 UTC by Jeff Hubbs
Modified: 2022-02-11 08:04 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jeff Hubbs 2021-01-04 23:26:53 UTC
Starting point was here: https://wiki.gentoo.org/wiki/PostgreSQL/QuickStart#Post-9.0_Migration. When you get to (this is as written in the QuickStart):

pg_upgrade -U postgres \
-d /var/lib/postgresql/8.4/data -D /var/lib/postgresql/9.3/data \
-b /usr/lib/postgresql-8.4/bin -B /usr/lib/postgresql-9.3/bin

you run into a proplem because the binaries aren't in /usr/lib/postgresql-X anymore; they're in /usr/lib64/postgresql-X/bin. 

BTW before I did this, I set POSTGRES_TARGETS="postgres12 postgres13" in make.conf.

So my pg_upgrade invocation became:

pg_upgrade -U postgres -d /var/lib/postgresql/12/data -D /var/lib/postgresql/13/data -b /usr/lib64/postgresql-12/bin -B /usr/lib64/postgresql-13/bin

but here's what resulted:

Performing Consistency Checks
-----------------------------
Checking cluster versions                                   ok
Checking database user is the install user                  ok
Checking database connection settings                       ok
Checking for prepared transactions                          ok
Checking for reg* data types in user tables                 ok
Checking for contrib/isn with bigint-passing mismatch       ok
Creating dump of global objects                             ok
Creating dump of database schemas
                                                            ok
Checking for presence of required libraries                 fatal

Your installation references loadable libraries that are missing from the
new installation.  You can add these libraries to the new installation,
or remove the functions using them from the old installation.  A list of
problem libraries is in the file:
    loadable_libraries.txt

Failure, exiting

And so I go look at that file:

$ cat /var/lib/postgresql/loadable_libraries.txt
could not load library "$libdir/postgis-2.5": ERROR:  could not access file "$libdir/postgis-2.5": No such file or directory
In database: commercial_bldgs
could not load library "$libdir/rtpostgis-2.5": ERROR:  could not access file "$libdir/rtpostgis-2.5": No such file or directory
In database: commercial_bldgs

At the moment those two libraries exist in /usr/lib64/postgresql-12/lib64 but not in /usr/lib64/postgresql-13/lib64/.

I've re-emerged PostGIS and PostgreSQL and it changed nothing.

NOTE: If I go into /usr/lib64/postgresql-13/lib64/ and make symlinks for postgis-2.5 and rtpostgis-2.5.so laterally into .../postgresql-12/lib64. the problem isn't "No such file or directory" but changes to "undefined symbol: elog_finish"



Reproducible: Always

Steps to Reproduce:
1. After having already emerged and configured postgresql-12 with postgis, follow https://wiki.gentoo.org/wiki/PostgreSQL/QuickStart#Post-9.0_Migration up to the pg_upgrade invocation.
2. Fix and invoke the pg_upgrade command based on what's in the aforementioned QuickStart
3. Observe the result with sadness.
Actual Results:  
$ pg_upgrade -U postgres -d /var/lib/postgresql/12/data -D /var/lib/postgresql/13/data -b /usr/lib64/postgresql-12/bin -B /usr/lib64/postgresql-13/bin
Performing Consistency Checks
-----------------------------
Checking cluster versions                                   ok
Checking database user is the install user                  ok
Checking database connection settings                       ok
Checking for prepared transactions                          ok
Checking for reg* data types in user tables                 ok
Checking for contrib/isn with bigint-passing mismatch       ok
Creating dump of global objects                             ok
Creating dump of database schemas
                                                            ok
Checking for presence of required libraries                 fatal

Your installation references loadable libraries that are missing from the
new installation.  You can add these libraries to the new installation,
or remove the functions using them from the old installation.  A list of
problem libraries is in the file:
    loadable_libraries.txt

$ cat /var/lib/postgresql/loadable_libraries.txt
could not load library "$libdir/postgis-2.5": ERROR:  could not access file "$libdir/postgis-2.5": No such file or directory
In database: commercial_bldgs
could not load library "$libdir/rtpostgis-2.5": ERROR:  could not access file "$libdir/rtpostgis-2.5": No such file or directory
In database: commercial_bldgs



Expected Results:  
pg_upgrade sshould have completed normally; https://wiki.gentoo.org/wiki/PostgreSQL/QuickStart#Post-9.0_Migration needs to be updated to reflect modern-day locations of binary files.


# emerge --info
Portage 3.0.12 (python 3.8.6-final-0, default/linux/amd64/17.1, gcc-9.3.0, glibc-2.32-r3, 5.4.80-gentoo-r1 x86_64)
=================================================================
System uname: Linux-5.4.80-gentoo-r1-x86_64-Pentium-R-_Dual-Core_CPU_E5300_@_2.60GHz-with-glibc2.2.5
KiB Mem:     4021396 total,   1161252 free
KiB Swap:    4194300 total,   4194300 free
Timestamp of repository gentoo: Mon, 04 Jan 2021 21:00:01 +0000
Head commit of repository gentoo: a9426af594c121fc32217227cddb3a58819192cf
sh bash 5.0_p18
ld GNU ld (Gentoo 2.34 p6) 2.34.0
app-shells/bash:          5.0_p18::gentoo
dev-lang/perl:            5.30.3::gentoo
dev-lang/python:          3.8.6::gentoo, 3.9.0::gentoo
dev-util/cmake:           3.17.4-r1::gentoo
sys-apps/baselayout:      2.7::gentoo
sys-apps/openrc:          0.42.1::gentoo
sys-apps/sandbox:         2.20::gentoo
sys-devel/autoconf:       2.69-r5::gentoo
sys-devel/automake:       1.16.2-r1::gentoo
sys-devel/binutils:       2.34-r2::gentoo
sys-devel/gcc:            9.3.0-r2::gentoo
sys-devel/gcc-config:     2.3.2-r1::gentoo
sys-devel/libtool:        2.4.6-r6::gentoo
sys-devel/make:           4.3::gentoo
sys-kernel/linux-headers: 5.4-r1::gentoo (virtual/os-headers)
sys-libs/glibc:           2.32-r3::gentoo
Repositories:

gentoo
    location: /var/db/repos/gentoo
    sync-type: rsync
    sync-uri: rsync://rsync.gentoo.org/gentoo-portage
    priority: -1000
    sync-rsync-verify-metamanifest: yes
    sync-rsync-verify-jobs: 1
    sync-rsync-verify-max-age: 24
    sync-rsync-extra-opts: 

ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="@FREE"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=native -O2 -pipe -fomit-frame-pointer"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/gconf /etc/gentoo-release /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-march=native -O2 -pipe -fomit-frame-pointer"
DISTDIR="/var/cache/distfiles"
ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR"
FCFLAGS="-march=native -O2 -pipe -fomit-frame-pointer"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-march=native -O2 -pipe -fomit-frame-pointer"
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LANG="C.UTF8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j3"
PKGDIR="/var/cache/binpkgs"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
USE="acl amd64 berkdb bzip2 cdr cli crypt dri fortran gdbm iconv ipv6 libglvnd libtirpc multilib ncurses nls nptl openmp pam pcre readline seccomp split-usr ssl tcpd unicode xattr zlib" ABI_X86="64" ADA_TARGET="gnat_2018" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="mmx mmxext sse sse2" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="libinput" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-2 php7-3 php7-4" POSTGRES_TARGETS="postgres12 postgres13" PYTHON_SINGLE_TARGET="python3_8" PYTHON_TARGETS="python2_7 python3_8" RUBY_TARGETS="ruby25 ruby26" USERLAND="GNU" VIDEO_CARDS="amdgpu fbdev intel nouveau radeon radeonsi vesa dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CC, CPPFLAGS, CTARGET, CXX, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, LINGUAS, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Comment 1 Jeff Hubbs 2021-01-05 00:10:21 UTC
Something else I noticed:

# psql --username=postgres --dbname=deleteme -c "CREATE EXTENSION postgis;"
ERROR:  could not open extension control file "/usr/share/postgresql-13/extension/postgis.control": No such file or directory

It's missing but present under /usr/share/postgresql-12/extension. It's as though the postgis installation just doesn't know anything about postgresql-13.
Comment 2 Jeff Hubbs 2021-01-05 00:42:42 UTC
I've found out more. It turns out that no version of postgis in portage, not even the ones that are still masked, are postgresql-13-ready. it seems that if you are using postgis on Gentoo, you have no choice but to mask out anything higher than postgresql-12.5 for the time being. Documentation-wise, this bridge is out.
Comment 3 Aaron W. Swenson gentoo-dev 2021-01-07 23:07:38 UTC
Well, the documentation mostly worked for me as is. There is a note about the /usr/lib.

I've just migrated a system yesterday using the instructions, and they mostly worked for me. There is a note about the /usr/lib on multilib systems. So, I needed to adjust for that. It does guide the reader that you need to if there's a hiccup, to read the log as you did. It should perhaps now mention updating the POSTGRES_TARGETS.

If you think there is a way to make it clearer, you are welcome to edit the documentation. It's a wiki. The worse that can happen to an edit done in earnest is for it to be reverted.

Also, the appropriate location to discuss edits, if you're uncertain about an edit, or even to start discussing trouble spots you have is at:
https://wiki.gentoo.org/wiki/Talk:PostgreSQL/QuickStart
Comment 4 Jeff Hubbs 2022-02-11 08:04:11 UTC
Thirteen months on and the situation is the same. I just revisited this problem on the same server and there hasn't been a new ebuild for PostgreSQL 13 in the interim although postgis has advanced from 2.5 to 3.1.3. 

Here is what happens now; it's slightly changed from before:

sh-5.1$ pg_upgrade -U postgres -d /var/lib/postgresql/12/data -D /var/lib/postgresql/13/data -b /usr/lib64/postgresql-12/bin -B /usr/lib64/postgresql-13/bin
Performing Consistency Checks
-----------------------------
Checking cluster versions                                   
This utility can only upgrade to PostgreSQL version 12.
Failure, exiting
sh-5.1$ pg_upgrade13 -U postgres -d /var/lib/postgresql/12/data -D /var/lib/postgresql/13/data -b /usr/lib64/postgresql-12/bin -B /usr/lib64/postgresql-13/bin
Performing Consistency Checks
-----------------------------
Checking cluster versions                                   ok
Checking database user is the install user                  ok
Checking database connection settings                       ok
Checking for prepared transactions                          ok
Checking for system-defined composite types in user tables  ok
Checking for reg* data types in user tables                 ok
Checking for contrib/isn with bigint-passing mismatch       ok
Creating dump of global objects                             ok
Creating dump of database schemas
  commercial_bldgs                                          
*failure*

Consult the last few lines of "pg_upgrade_dump_17993.log" for
the probable cause of the failure.
Failure, exiting

That log file says:

command: "/usr/lib64/postgresql-13/bin/pg_dump" --host /var/lib/postgresql --port 50432 --userna
me postgres --schema-only --quote-all-identifiers --binary-upgrade --format=custom  --file="pg_u
pgrade_dump_17993.custom" 'dbname=commercial_bldgs' >> "pg_upgrade_dump_17993.log" 2>&1
pg_dump: error: query failed: ERROR:  could not access file "$libdir/postgis-2.5": No such file 
or directory

PostgreSQL and PostGIS have both been reemerged with POSTGRES_TARGETS="postgres12 postgres13" in make.conf. Note that I am using pg_upgrade13 now. 

For some reason the pg_update process is still looking for $libdir/postgis-2.5 whem it seems like it should be looking for postgis-3.