P4P on Windows loads ca.dll on import, breaking pyepics

[newville]

On Windows, at least the current PyPI version, import p4p with

import p4p

or

from p4p.client.thread import Context

silently import ca.dll from epicscorelibs. Since there can only be one shared library per process, this breaks any downstream library that would also import ca.dll. By default, pypepics bundles and uses ca.dll, so that if p4p is imported as above, that script is likely to fail to use pyepics.

Note 1: loading of ca shared library does not happen at import-time on Linux/macOS, making the behaviour of p4p inconsistent.

Note 2: this is not an issue with epicscorelibs or pvxslibs, only with p4p:

>>> import epicscorelibs, epicscorelibs.path, pvxlibs.path, numpy
>>> import epics
>>> epics.caget('TestPV.VAL')

works as expected and does not give the error "loading CA DLL failed".

I think that P4P should not load CA.dll at import time, and it should work consistently on all platforms.

On all platforms, pyepics supports setting the location of LIBCA after import but before any real work is done. It supports using the environment variable PYPEPICS_LIBCA to specify the location of LIBCA. If this is not set, it will use the ones it installs for itself. We can work around this error in P4P, but you should fix this and not import LIBCA at import p4p.

[mdavidsaver]

silently import ca.dll from epicscorelibs

Hello @newville, what error are you seeing which leads you to this conclusion? On what platform(s)? I do not see this, at least on Debian 13 Linux/amd64, cpython 3.13 with

$ ./p4pdev/bin/pip freeze
epicscorelibs==7.0.10.99.0.1
nose2==0.16.0
numpy==2.4.6
p4p==4.2.2
ply==3.11
pvxslibs==1.5.1
setuptools==82.0.1
setuptools_dso==2.12.3

$ ./p4pdev/bin/python
Python 3.13.5 (main, May  5 2026, 21:05:52) [GCC 14.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from p4p.client.thread import Context
>>> import os
>>> os.getpid()
171095
>>> 

Inspecting /proc/171095/maps shows only libCom from base. This matches my expectation as the compiled extension of P4P only loads libpvxs.so, which in turn only depends on libCom.so from epics-base.

$ ldd python3.13d/linux-x86_64/p4p/_p4p.so 
        linux-vdso.so.1 (0x00007fe658cd6000)
        libpvxs.so.1.5 => /home/dev/epics/p4p/python3.13d/linux-x86_64/p4p/../../../../pvxs/lib/linux-x86_64/libpvxs.so.1.5 (0x00007fe658a5e000)
        libCom.so.3.25.1 => /home/dev/epics/p4p/python3.13d/linux-x86_64/p4p/../../../../base-git/lib/linux-x86_64/libCom.so.3.25.1 (0x00007fe6589e2000)
        libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fe658600000)
        libgcc_s.so.1 => /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fe6589b3000)
        libc.so.6 => /usr/lib/x86_64-linux-gnu/libc.so.6 (0x00007fe65840c000)
        libevent_openssl-2.1.so.7 => /lib/x86_64-linux-gnu/libevent_openssl-2.1.so.7 (0x00007fe658975000)
        libevent_core-2.1.so.7 => /lib/x86_64-linux-gnu/libevent_core-2.1.so.7 (0x00007fe65893f000)
        libevent_pthreads-2.1.so.7 => /lib/x86_64-linux-gnu/libevent_pthreads-2.1.so.7 (0x00007fe65893a000)
        libssl.so.3 => /lib/x86_64-linux-gnu/libssl.so.3 (0x00007fe6582fe000)
        libcrypto.so.3 => /lib/x86_64-linux-gnu/libcrypto.so.3 (0x00007fe657c00000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fe658cd8000)
        libreadline.so.8 => /lib/x86_64-linux-gnu/libreadline.so.8 (0x00007fe6588de000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fe657b10000)
        libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fe6588be000)
        libzstd.so.1 => /lib/x86_64-linux-gnu/libzstd.so.1 (0x00007fe657a46000)
        libtinfo.so.6 => /lib/x86_64-linux-gnu/libtinfo.so.6 (0x00007fe658886000)

[mdavidsaver]

If I had to make a guess, it would be that while an error manifests when loading libca.so (or ca.dll?), the root cause is a duplicate symbol when indirectly loading a second (almost) copy of libCom.so. Which can not be avoided on all targets.

[mdavidsaver]

fyi. On Linux, the glibc specific environment variable $LD_DEBUG can be helpful when troubleshooting loading errors. see man ld.so.

[newville]

@mdavidsaver

. Hello @newville, what error are you seeing which leads you to this conclusion? On what platform(s)?
I do not see this, at least on Debian 13 Linux/amd64, cpython 3.13 with

As I stated clearly in both the label for this Issue and the first sentence of the report, the platform affected is Windows.
I do not see this issue on Linux or macOS. This is exactly about Microsoft Windows. I referred to ca.dll precisely.

The error seen is from pyepics -- not from p4p -- complaining that it cannot load ca.dll, as the process has already loaded ca.dll somewhere in

import p4p

but not in the importing of epicscorelibs or pvxslibs. To be super-clear:

>>> import epicscorelibs, epicscorelibs.path, pvxslibs.path, numpy
>>> import epics
>>> epics.caget('TestPV.VAL')

succeeds, while

>>> import epicscorelibs, epicscorelibs.path, pvxslibs.path, numpy
>>> import p4p
>>> import epics
>>> epics.caget('TestPV.VAL')

fails.

Again: On Windows.

see man ld.so

Sheesh.

[mdavidsaver]

The error seen is from pyepics -- not from p4p -- complaining that it cannot load ca.dll,

as the process has already loaded ca.dll somewhere in

I believe that this is not the case. As stated above, P4P does not cause ca.dll to be loaded. I believe the error you mention comes when loading Com.dll, which is implicitly loaded when ca.dll is requested.

I do not see this issue on Linux or macOS

These *NIX targets have the RTLD_LOCAL option for dlopen(), which allows multiple versions of a shared object to be loaded, but isolated from each other. Python ctypes sets this flag by default when available. I do not believe that Windows has an equivalent. Meaning that on Windows it will not be possible to load two versions of Com.dll into the same process.

[newville]

@mdavidsaver

Case 1: Without p4p, the default usage of pyepics

(base) C:\Users\xas_user\Documents\GitHub>python
Python 3.13.10 | packaged by conda-forge | (main, Dec  3 2025, 09:03:25) [MSC v.1944 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import ctypes
>>> import epics
>>> com_dllname = epics.ca.find_libCom()
>>> ca_dllname = epics.ca.find_libca()
>>> print(f"{com_dllname=}")
com_dllname='C:\\Users\\xas_user\\AppData\\Local\\xraylarch\\Lib\\site-packages\\epics\\clibs\\win64\\Com.dll'
>>> print(f"{ca_dllname=}")
ca_dllname='C:\\Users\\xas_user\\AppData\\Local\\xraylarch\\Lib\\site-packages\\epics\\clibs\\win64\\ca.dll'
>>>
>>> libcom = ctypes.windll.LoadLibrary(com_dllname)
>>> print(f"{libcom=}")
libcom=<WinDLL 'C:\Users\xas_user\AppData\Local\xraylarch\Lib\site-packages\epics\clibs\win64\Com.dll',
handle 7ffe2c090000 at 0x1c2a5dd9d10>
>>> libca = ctypes.windll.LoadLibrary(ca_dllname)
>>> print(f"{libca=}")
libca=<WinDLL 'C:\Users\xas_user\AppData\Local\xraylarch\Lib\site-packages\epics\clibs\win64\ca.dll', handle 7ffe2c030000 at 0x1c2a5da9ba0>
>>> epics.caget('13IDA:m1.VAL')
-0.27296018499999963

works as expected.

Case 2: Adding import p4p:

base) C:\Users\xas_user\Documents\GitHub>python
Python 3.13.10 | packaged by conda-forge | (main, Dec  3 2025, 09:03:25) [MSC v.1944 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import p4p
>>> import epics
>>> import ctypes
>>> com_dllname = epics.ca.find_libCom()
>>> ca_dllname = epics.ca.find_libca()
>>> print(f"{com_dllname=}")
com_dllname='C:\\Users\\xas_user\\AppData\\Local\\xraylarch\\Lib\\site-packages\\epics\\clibs\\win64\\Com.dll'
>>> print(f"{ca_dllname=}")
ca_dllname='C:\\Users\\xas_user\\AppData\\Local\\xraylarch\\Lib\\site-packages\\epics\\clibs\\win64\\ca.dll'
>>> libcom = ctypes.windll.LoadLibrary(com_dllname)
>>>
>>> print(libcom)
<WinDLL 'C:\Users\xas_user\AppData\Local\xraylarch\Lib\site-packages\epics\clibs\win64\Com.dll', handle 7ffe2bb10000 at 0x16b5c030690>
>>> libca = ctypes.windll.LoadLibrary(ca_dllname)
Traceback (most recent call last):
  File "<python-input-12>", line 1, in <module>
    libca = ctypes.windll.LoadLibrary(ca_dllname)
  File "C:\Users\xas_user\AppData\Local\xraylarch\Lib\ctypes\__init__.py", line 480, in LoadLibrary
    return self._dlltype(name)
           ~~~~~~~~~~~~~^^^^^^
  File "C:\Users\xas_user\AppData\Local\xraylarch\Lib\ctypes\__init__.py", line 361, in __init__
    self._handle = self._load_library(name, mode, handle, winmode)
                   ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\xas_user\AppData\Local\xraylarch\Lib\ctypes\__init__.py", line 379, in _load_library
    return _LoadLibrary(self._name, winmode)
OSError: [WinError 127] The specified procedure could not be found

fails. It tuns out that it succeeded at loading libCom, and failed loading libca.
But, for sure, the simple addition of import p4p is clearly messing with the loading of libca.

Case 3: still use import p4p, but use ca.dll from epicsorelibs:

(base) C:\Users\xas_user\Documents\GitHub>python
Python 3.13.10 | packaged by conda-forge | (main, Dec  3 2025, 09:03:25) [MSC v.1944 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import p4p
>>> import epicscorelibs
>>> import os
>>> import epics
>>> import ctypes
>>> os.environ['PYEPICS_LIBCA'] = epicscorelibs.lib.ca_dsoinfo.filename
>>> ca_dllname = epics.ca.find_libca()
>>> print(ca_dllname)
C:\Users\xas_user\AppData\Local\xraylarch\Lib\site-packages\epicscorelibs\lib\ca.dll
>>> libca = ctypes.windll.LoadLibrary(ca_dllname)
>>> print(libca)
<WinDLL 'C:\Users\xas_user\AppData\Local\xraylarch\Lib\site-packages\epicscorelibs\lib\ca.dll', handle 7ffe2c3d0000 at 0x2086aac47d0>
>>> epics.caget('13IDA:m1.VAL')
-0.27296018499999963

works. So, whatever p4p is doing, it is placing expectations of what ca.dll is loaded.

Because pyepics waits to loads the libca and libCom only just before the first real call to the CA library is needed, we can allow other CA libraries to be used, and work around whatever p4p is doing to cause this error.

That is a workaround, and does not change the fact that import p4p is causing other libraries to be unable to load ca.dll.

Feel free to close this issue if you find that it is not something that you are able or willing to resolve.

[mdavidsaver]

Hello Matt,

ctypes.windll.LoadLibrary() does not work the way one might hope. The returned WinDLL object __repr__ will always show the path passed as an argument, regardless of what the underlying LoadLibraryExW() C call does (not) do.

A better test uses GetModuleHandleA() which looks up in the process memory map without loading additional DLLs.

Try the following:

from ctypes import *
print('self', windll.kernel32.GetModuleHandleA(None))
print('pvxs', windll.kernel32.GetModuleHandleA(b'pvxs.dll'))
print('ca', windll.kernel32.GetModuleHandleA(b'ca.dll'))
from p4p.client.thread import Context
print('pvxs', windll.kernel32.GetModuleHandleA(b'pvxs.dll'))
print('ca', windll.kernel32.GetModuleHandleA(b'ca.dll'))

On a GHA windows-2025 runner with cpy 3.13 yields:

self -242089984
pvxs 0
ca 0
pvxs -1939668992
ca 0

This shows that importing P4P has loaded pvxs.dll, but not ca.dll.

As a note: py 3.14 adds a wrapper ctypes.util.dllist() for iterating loaded libraries.

[newville]

@mdavidsaver Hm... seems weird. Something about importing p4p prevents an otherwise working version of ca.dll from being loaded. The message from ctypes is not very informative about which procedure is missing.

Note that Com.dll was successfully loaded. And other versions of ca.dll (at least, the version from epicscorelibs) can be loaded after import p4p and work fine.

The Com.dll and ca.dll for Win64 distributed with pyepics are from 2021. That doesn't seem that old to me, but maybe there is some conflict with assumptions here. This is the first issue I've encountered or heard about with these. I think it might be the first time I've heard of libca failing to load on any system when there is not already another version of that library loaded. I suppose that this could be an error in ctypes, but it seems more likely that something in import p4p -- perhaps the loading of the not-completely-unrelated pvxs.dll? -- is somehow causing the conflict, especially if other versions of ca.dll can be loaded.

At pyepics, we have pushed a fix (pyepics/pyepics#285): if epicscorelibs is in sys.modules at the time when pyepics is initializing libca, its libca will be used. pyepics 3.5.10 has this fix enabled, so that p4p and pyepics can be used in the same script.

But, I can imagine that others might run into similar problems.