error while loading shared libraries: cannot open shared object file

2026-10-25 By luckyray Debian 12 · Ubuntu 24.04 · Alpine 3.20

$ ./report
./report: error while loading shared libraries: libssl.so.1.1: cannot open shared object file:
No such file or directory

The binary started, so the kernel accepted it. The failure happened a few microseconds later, inside ld.so. That distinction is the whole diagnosis: this is a loader problem, not a file problem.

Three messages that look alike and mean different things

Message	Emitted by	Meaning
`cannot execute binary file: Exec format error`	kernel	wrong architecture — the loader never ran
`No such file or directory` (about the binary)	kernel	`PT_INTERP` points at an interpreter that does not exist, e.g. a glibc binary on Alpine
`error while loading shared libraries: X: cannot open shared object file`	`ld.so`	the interpreter exists and ran; one dependency could not be located

The middle case is the one that wastes the most time, because the missing file is not the binary. It is /lib64/ld-linux-x86-64.so.2, which does not exist on a musl system. If ./app reports “no such file or directory” while ls -l ./app clearly shows the file, check the interpreter with readelf -l ./app | grep interpreter before doing anything else.

How the loader searches

For each entry in the binary’s DT_NEEDED list, ld.so walks a fixed set of locations:

DT_RPATH from the binary — only if DT_RUNPATH is absent. Searched before the environment.
LD_LIBRARY_PATH
DT_RUNPATH — searched after the environment, which is the opposite of RPATH.
/etc/ld.so.cache — the index built by ldconfig.
/lib, /usr/lib and the multiarch variants (/usr/lib/x86_64-linux-gnu).

Two consequences follow, and both explain real incidents:

A stale RPATH shadows the library you just exported via LD_LIBRARY_PATH. You export the correct path, run the binary, and it still loads the old one from the RPATH directory. People conclude the environment variable “does not work”.
On modern glibc, LD_LIBRARY_PATH beats RUNPATH. On a binary that uses RPATH, it does not. Identical-looking build configurations therefore behave differently depending on which option the linker was given (--enable-new-dtags switches the default).

Why “installed” and “found” are different

The loader searches for a filename, not for a package, a version, or a purpose.

$ dpkg -l | grep openssl
ii  openssl   3.0.13-0ubuntu3.4   amd64   Secure Sockets Layer toolkit

$ ls /usr/lib/x86_64-linux-gnu/libssl.so*
/usr/lib/x86_64-linux-gnu/libssl.so.3

OpenSSL is installed. libssl.so.1.1 is not present, because it is a different SONAME belonging to a different major version. There is no symlink that would make the loader accept .so.3 for a DT_NEEDED entry of .so.1.1 — and there should not be. ABI compatibility between OpenSSL 1.1 and 3.0 does not exist, which is precisely why the SONAME changed.

Common variants of the same confusion:

Multiarch path drift. The library is in /usr/lib/x86_64-linux-gnu/ but the binary’s RUNPATH only lists /usr/lib. Both exist; only one contains the file.
Stale ldconfig cache. You copied a library into /usr/local/lib and skipped ldconfig -v. The directory is in the cache list, but the cache does not yet know about the file.
Container rootfs. The library exists on the host. The container never sees it. This is the entire point of containers, and it is still a frequent surprise when someone mounts a binary into a scratch or distroless image, which by design contains no ld.so and no libc.
chroot. Same mechanism, different decade.

Diagnosing it properly

$ readelf -d ./report | grep -E 'NEEDED|RPATH|RUNPATH'
 0x0000000000000001 (NEEDED)   Shared library: [libssl.so.1.1]
 0x0000000000000001 (NEEDED)   Shared library: [libc.so.6]
 0x000000000000001d (RUNPATH)  Library runpath: [$ORIGIN/../lib]

$ ldconfig -p | grep libssl
	libssl.so.3 (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/libssl.so.3

$ LD_DEBUG=libs ./report 2>&1 | grep -A2 'libssl'
     21490:	find library=libssl.so.1.1 [0]; searching
     21490:	  tried file=/opt/app/../lib/libssl.so.1.1
     21490:	search path=$ORIGIN/../lib  (RUNPATH from file ./report)

LD_DEBUG=libs is the tool people forget. It prints every directory actually searched, which ends the guesswork immediately.

On ldd: it is a shell script that runs the loader in a special mode, and on some systems it can execute the binary’s dependencies. Never run it against a file you do not trust. The safe equivalent is objdump -p ./report | grep NEEDED, which only reads. The Dependency Doctor does the same parse in a browser and can cross-check pasted ldd output against the NEEDED list.

The fixes, in order of preference

1. Install the package that provides the exact filename. This is the correct fix 90% of the time, because the version mismatch is real and the binary genuinely needs that ABI.

apt-file search libssl.so.1.1          # Debian/Ubuntu: find the package
dnf provides '*/libssl.so.1.1'         # RHEL family

On current distributions the answer is often “this package no longer exists”, which means the fix is to rebuild against the distribution’s OpenSSL 3, not to hunt down a legacy .deb.

2. Rebuild against the target environment. The durable answer. Building in the same base image the software runs in removes both the library and the GLIBC class of problems at once.

3. Ship the libraries next to the binary and use $ORIGIN. Correct and portable, and the only sane approach for a vendor-distributed binary that must run on many distributions:

patchelf --set-rpath '$ORIGIN/lib' ./report

$ORIGIN expands to the directory containing the binary, so the result is relocatable. Prefer RUNPATH (the default for --set-rpath on modern patchelf) over RPATH, since RUNPATH loses to LD_LIBRARY_PATH and therefore does not break local overrides.

4. LD_LIBRARY_PATH for debugging only. It is inherited by every child process, it silently changes which library every tool in that shell loads, and it is ignored for setuid binaries. Use it to prove the diagnosis, then remove it.

5. Symlink the SONAME. ln -s libssl.so.3 libssl.so.1.1 will make the error disappear and the program crash later, or worse, run with subtly wrong behaviour. Do not do this in production. If you have already done it, remove it — a wrong-ABI load is far harder to debug than a missing library.

If the binary starts and then aborts with version 'GLIBC_2.34' not found, the library was found and the loader rejected its symbol versions. That is a different problem with a different fix — covered in GLIBC version not found.

Every command in this guide was run against the distribution named in the output. If your output differs, the mechanism section explains why — and the ELF Inspector shows the same fields without needing a shell.