Add Device Type Components - Module Bays and Console Ports

[kennethdammyr]

Hi

When updating a device with the script add_device_type_components.py, both interfaces, module bays and console ports are populated. However, the tab menues for module bays and console ports does not get updated with the new data. In this screenshot, you may see how the device has got 17 module bays, but only showing 1 (which was there from before) in the menu:

If there are no module bays or console ports, the tab menu item is not displayed. Hence you will not have the oportunity to use them. As shown here:

Interfaces works as expected.
Any idea what could cause this?

Netbox v3.6.6

[BriacD]

I just noticed this issue with PDUs and adding Power Outlets. Have you come across a solution? Script works great and does exactly what I've been needing but this is making it hard to move forward with all the changes I need to make.

[UniverseSandbox]

Same Issue here. Does someone have a solution?

[BriacD]

I upgraded to the latest version and that fixed my issue.

[ip-rx]

This is also affecting me. v3.7.4

Same issue. Ran script and created missing components (Interfaces, Console Port, PSU Module bays). Only Interfaces are accessible directly via the device's page. To find Console or PSU Mod Bays, I have to search them up in components list.

Not breaking, but certainly an annoyance.

With console port from script only

Added a dummy console port

[ip-rx]

@candlerb found a fix and already submitted a PR for it

#101

my team has tested this and it works.

[ip-rx]

The next issue with this bug:

Any devices that were created and affected by this bug, will still have an incorrect count of objects.

device.consoleports.count() (actual) and device.console_port_count (meta) will be inconsistent. The fix to the script calls a .save() which engages the function utilities.counters.update_counter(), but the bug caused device.console_port_count to be permanently wrong.

I threw together a script that just calcs the delta and feeds the inputs into utilities.counters.update_counter(). You can run it against any device pretty safely as it gets the literal count of each component and feeds the correct delta (positive or negative integer) into update_counter()

This logic could easily be inserted into the main script as a way to fix the issue retroactively when ran on a device.

"""
This script runs a counter check on each device's components and updates discrepancies
"""

from dcim.models import Manufacturer, DeviceType, Device
from extras.scripts import Script, ObjectVar, MultiObjectVar
from utilities.counters import update_counter


class FixDeviceComponentCounts(Script):
    class Meta:
        name = "Fix Device Component Counts"
        description = (
            "run a counter check on each device's components and updates discrepancies"
        )

    manufacturer = ObjectVar(
        model=Manufacturer,
        required=False,
    )
    device_type = ObjectVar(
        model=DeviceType,
        query_params={
            "manufacturer_id": "$manufacturer",
        },
        required=False,
    )
    devices = MultiObjectVar(
        model=Device,
        query_params={
            "device_type_id": "$device_type",
        },
    )

    def run(self, data, commit):
        for device in data["devices"]:
            device.full_clean()
            for counter_name, count_attr_name in [
                ("console_port_count", "consoleports"),
                ("console_server_port_count", "consoleserverports"),
                ("power_port_count", "powerports"),
                ("power_outlet_count", "poweroutlets"),
                ("interfaces_count", "interfaces"),
                ("rear_port_count", "rearports"),
                ("front_port_count", "frontports"),
                ("device_bay_count", "devicebays"),
                ("module_bay_count", "modulebays"),
            ]:
                actual_attr_count = getattr(device, count_attr_name).count()
                expected_attr_count = getattr(device, counter_name)
                # delta = device.consoleports.count() - device.console_port_count
                delta = actual_attr_count - expected_attr_count
                if delta != 0:
                    update_counter(device._meta.model, device.pk, counter_name, delta)

                    self.log_success(
                        "%s (%d): updated %s from %d to %d"
                        % (
                            device.name,
                            device.id,
                            counter_name,
                            expected_attr_count,
                            actual_attr_count,
                        )
                    )

Device with incorrect counts

Execution of script above

Same device with corrected counts

[kkthxbye-code]

@ip-rx - there's a management command for that.

https://github.com/netbox-community/netbox/blob/develop/netbox/utilities/management/commands/calculate_cached_counts.py

[candlerb]

That's the right answer, or more succinctly:

/opt/netbox/venv/bin/python /opt/netbox/netbox/manage.py calculate_cached_counts

will fix the problem.

[ip-rx]

Even better, thanks!

[jburgon]

When I run this script I get an error about the module bays. Anyone know what this might mean? If I comment out the line for module bays then it works. Obviously module bays don't get applied but everything else does.

`An exception occurred: IntegrityError: null value in column "level" of relation "dcim_modulebay" violates not-null constraint DETAIL: Failing row contains (2024-09-20 18:40:03.531782+00, 2024-09-20 18:40:03.531805+00, {}, 261, PIM1, PIM00000001, Mini-PIM 1, PIM1, , 118, null, null, null, null, null, null).

Traceback (most recent call last):
File "/opt/netbox/venv/lib64/python3.11/site-packages/django/db/backends/utils.py", line 105, in _execute
return self.cursor.execute(sql, params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/netbox/venv/lib64/python3.11/site-packages/psycopg/cursor.py", line 97, in execute
raise ex.with_traceback(None)
psycopg.errors.NotNullViolation: null value in column "level" of relation "dcim_modulebay" violates not-null constraint
DETAIL: Failing row contains (2024-09-20 18:40:03.531782+00, 2024-09-20 18:40:03.531805+00, {}, 261, PIM1, PIM00000001, Mini-PIM 1, PIM1, , 118, null, null, null, null, null, null).

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/opt/netbox/netbox/extras/jobs.py", line 45, in run_script
script.output = script.run(data, commit)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/netbox/netbox/scripts/add_device_type_components.py", line 66, in run
klass.objects.bulk_create(items)
File "/opt/netbox/venv/lib64/python3.11/site-packages/django/db/models/manager.py", line 87, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/netbox/venv/lib64/python3.11/site-packages/django/db/models/query.py", line 835, in bulk_create
returned_columns = self._batched_insert(
^^^^^^^^^^^^^^^^^^^^^
File "/opt/netbox/venv/lib64/python3.11/site-packages/django/db/models/query.py", line 1875, in _batched_insert
self._insert(
File "/opt/netbox/venv/lib64/python3.11/site-packages/django/db/models/query.py", line 1847, in _insert
return query.get_compiler(using=using).execute_sql(returning_fields)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/netbox/venv/lib64/python3.11/site-packages/django/db/models/sql/compiler.py", line 1823, in execute_sql
cursor.execute(sql, params)
File "/opt/netbox/venv/lib64/python3.11/site-packages/django/db/backends/utils.py", line 79, in execute
return self._execute_with_wrappers(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/netbox/venv/lib64/python3.11/site-packages/django/db/backends/utils.py", line 92, in _execute_with_wrappers
return executor(sql, params, many, context)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/netbox/venv/lib64/python3.11/site-packages/django/db/backends/utils.py", line 100, in _execute
with self.db.wrap_database_errors:
File "/opt/netbox/venv/lib64/python3.11/site-packages/django/db/utils.py", line 91, in exit
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/opt/netbox/venv/lib64/python3.11/site-packages/django/db/backends/utils.py", line 105, in _execute
return self.cursor.execute(sql, params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/netbox/venv/lib64/python3.11/site-packages/psycopg/cursor.py", line 97, in execute
raise ex.with_traceback(None)
django.db.utils.IntegrityError: null value in column "level" of relation "dcim_modulebay" violates not-null constraint
DETAIL: Failing row contains (2024-09-20 18:40:03.531782+00, 2024-09-20 18:40:03.531805+00, {}, 261, PIM1, PIM00000001, Mini-PIM 1, PIM1, , 118, null, null, null, null, null, null).`

[candlerb]

I guess it's because the module model changed in Netbox v4.1.0 to allow nested (hierarchical) modules.

[joekripal]

I'm running into the same error as @jburgon. I tried to piece together a fix don't seem to be able to.

[rizlas]

@ip-rx - there's a management command for that.

https://github.com/netbox-community/netbox/blob/develop/netbox/utilities/management/commands/calculate_cached_counts.py

That's the right answer, or more succinctly:

/opt/netbox/venv/bin/python /opt/netbox/netbox/manage.py calculate_cached_counts

will fix the problem.

Dunno where if you found this, but you are legend. Thank you!

Just for ref another issue: netbox-community/netbox#14446

[natecohen]

When I run this script I get an error about the module bays. Anyone know what this might mean? If I comment out the line for module bays then it works. Obviously module bays don't get applied but everything else does.

`An exception occurred: IntegrityError: null value in column "level" of relation "dcim_modulebay" violates not-null constraint DETAIL: Failing row contains (2024-09-20 18:40:03.531782+00, 2024-09-20 18:40:03.531805+00, {}, 261, PIM1, PIM00000001, Mini-PIM 1, PIM1, , 118, null, null, null, null, null, null).

EDIT: Based on yaiqsa's comment I have modified my script after realizing I had duplicate logic, removing bulk_create() solves both issues of modulebays not adding and component counts not updating.

I also added a new option to disable adding interfaces since this works better with my discovery scripts which populate interface data.

https://gist.github.com/natecohen/85f02279633550aca0c6e7426c7a9e9b