Add ibm-db package to enable DB2 as datasource:

[nicof38]

What type of PR is this?

• Refactor
• Feature
• Bug Fix
• New Query Runner (Data Source)
• New Alert Destination
• Other

Description

No modification for compatiblity, just re-enable the DB2 driver

How is this tested?

• Unit tests (pytest, jest)
• E2E Tests (Cypress)
• Manually
• N/A

Add a datasource, DB2 datasource is available and working as it was before release 25.

Related Tickets & Documents

#7349 (comment)

Mobile & Desktop Screenshots/Recordings (if there are UI changes)

[arikfr]

In 65d0eb7 we commented out the ibm-db2 package because it seemed to have issues on Apple Silicon - did the newer version of the package resolve this?

[nicof38]

I can find at least this one for M1 : ibmdb/python-ibmdb#712 . Does it match with what you had observed ?

[nicof38]

In 65d0eb7 we commented out the ibm-db2 package because it seemed to have issues on Apple Silicon - did the newer version of the package resolve this?

There is also this one : https://github.com/ibmdb/python-ibmdb/issues/797#issuecomment-2565727152

[yoshiokatsuneo]

@nicof38

There is also this one : https://github.com/ibmdb/python-ibmdb/issues/797#issuecomment-2565727152

Somehow, the URL is correct but the link was broken.

[yoshiokatsuneo]

And, when I try to build Redash on my Apple Silicon Mac using "make up", I got the following error.

 > [scheduler stage-1  9/12] RUN /etc/poetry/bin/poetry install --only main,all_ds,dev --no-root --no-interaction --no-ansi:
...
21.85 PEP517 build of a dependency failed
21.85 
21.85 Backend subprocess exited when trying to invoke get_requires_for_build_wheel
21.85 
21.85     | Command '['/tmp/tmp89wf3juj/.venv/bin/python', '/etc/poetry/venv/lib/python3.10/site-packages/pyproject_hooks/_in_process/_in_process.py', 'get_requires_for_build_wheel', '/tmp/tmp8h3_jxe6']' returned non-zero exit status 1.
21.85     | 
21.85     | Detected 64-bit Python
21.85     | Detected platform = linux, uname = aarch64
21.85     | Traceback (most recent call last):
21.85     |   File "/etc/poetry/venv/lib/python3.10/site-packages/pyproject_hooks/_in_process/_in_process.py", line 389, in <module>
21.85     |     main()
21.85     |   File "/etc/poetry/venv/lib/python3.10/site-packages/pyproject_hooks/_in_process/_in_process.py", line 373, in main
21.85     |     json_out["return_val"] = hook(**hook_input["kwargs"])
21.85     |   File "/etc/poetry/venv/lib/python3.10/site-packages/pyproject_hooks/_in_process/_in_process.py", line 143, in get_requires_for_build_wheel
21.85     |     return hook(config_settings)
21.85     |   File "/tmp/tmp89wf3juj/.venv/lib/python3.10/site-packages/setuptools/build_meta.py", line 331, in get_requires_for_build_wheel
21.85     |     return self._get_build_requires(config_settings, requirements=[])
21.85     |   File "/tmp/tmp89wf3juj/.venv/lib/python3.10/site-packages/setuptools/build_meta.py", line 301, in _get_build_requires
21.85     |     self.run_setup()
21.85     |   File "/tmp/tmp89wf3juj/.venv/lib/python3.10/site-packages/setuptools/build_meta.py", line 317, in run_setup
21.85     |     exec(code, locals())
21.85     |   File "<string>", line 380, in <module>
21.85     | NameError: name 'arch_' is not defined
21.85 
21.85 Note: This error originates from the build backend, and is likely not a problem with poetry but one of the following issues with ibm-db (3.2.7)
21.85 
21.85   - not supporting PEP 517 builds
21.85   - not specifying PEP 517 build requirements correctly
21.85   - the build requirements are incompatible with your operating system or Python version
21.85   - the build requirements are missing system dependencies (eg: compilers, libraries, headers).
21.85 
21.85 You can verify this by running pip wheel --no-cache-dir --use-pep517 "ibm-db (==3.2.7)".
...

[nicof38]

@nicof38

There is also this one : https://github.com/ibmdb/python-ibmdb/issues/797#issuecomment-2565727152

Somehow, the URL is correct but the link was broken.

yes, correct is ibmdb/python-ibmdb#797 (comment)

[nicof38]

name 'arch_' is not defined

it should be linked to this : ibmdb/python-ibmdb#712
need to install python x64 instead of python arm64, so python intel version.

@yoshiokatsuneo is this something that may be done ? at least tried on M1 to confirm it fixes this issue ?

[yoshiokatsuneo]

@nicof38 I think running intel binary on Apple Silicon requires intel CPU emulation that is too slow to be practical solution...

[nicof38]

@nicof38 I think running intel binary on Apple Silicon requires intel CPU emulation that is too slow to be practical solution...

even without intel, it should work as mentionned in readme here : https://github.com/ibmdb/python-ibmdb/blob/master/README.md#for-macos-m1m2-apple-silicon-chip-system

Can you test it on your side with your M1 ?

[yoshiokatsuneo]

As I mentioned, I tried to use this PR with ibm-db (3.2.7) , but the installing ibm-db failed...

#7581 (comment)

[nicof38]

@yoshiokatsuneo ok, ibm-db package only only state to run on python x64 for M1, which is not the case here, so not working for this platform.
Also, I did not saw anything mentionning that the build platform should be M1 compatible ? Have I missed something ? If multiple build platform have to be supported, it should be good to have a complete list and include include it in the developper section, but it may highly reduce the number of contributor if we have to stick to multiple platform

[yoshiokatsuneo]

even without intel, it should work as mentionned in readme here : https://github.com/ibmdb/python-ibmdb/blob/master/README.md#for-macos-m1m2-apple-silicon-chip-system

And, I just noticed that the documentation says about MacOS on Apple Silicon. But Redash on Mac runs on Docker, so actually it requires to run on Linux on Apple Silicon(ARM), instead of MacOS on Apple Silicon.

[yoshiokatsuneo]

Also, I did not saw anything mentionning that the build platform should be M1 compatible ?

It looks right. there is no official description mentioning about CPU...
There is no description that we support ARM.
And, there is no description that we support Intel.
It may be nice to have the description.

At the same time, nowadays, I feel practically most open source software support ARM and Intel nowadays, and it is common to support both platform nowadays...

Technically, it should be possible to support both platform, and make ibm-db2 available only on intel platform...

[yoshiokatsuneo]

Possibly, we may enable ibm-db only on intel platform on pyproject.tom like below ?

ibm-db = { version = "^3.2.7", markers = "platform_machine == 'x86_64' or platform_machine == 'AMD64'" }

(I only tested on Mac.)

[nicof38]

@yoshiokatsuneo thanks for this one, tested on my side, and working fine also. Gonna modify the PR with this ;)

[yoshiokatsuneo]

@arikfr @eradman

I think this looks OK.
But, I would like to confirm as it introduce platform specific packages.
May I hear your idea ?

[eradman]

Tested on aarch64, and observed that DB2 does not show up as an option when adding a new Data Source. (it does on x86_64) I think this is okay since this is a proprietary database.

[arikfr]

Enabling only on Intel platform feels reasonable, although it still means that by default it won't be available.

[cubic-dev-ai]

No issues found across 2 files

[yoshiokatsuneo]

@nicof38 @eradman @arikfr

As it looks this PR is reasonable solution for stakeholders, I approve and merge the PR.
Thank you !

[nicof38]

The poetry install is running fine on my side, and was OK also for you @yoshiokatsuneo . Any idea why not building at this time ?

[yoshiokatsuneo]

@nicof38

Some test just sometimes fails...
I think @eradman rerun the test and it succeeded and merged now. Thank you !