Unlock support for CircuitPython

[simonprickett]

CircuitPython is another very popular runtime for Python on microcontrollers, especially Adafruit products. Figure out how to get the driver working in that environment and provide install instructions and example code for it.

[simonprickett]

I've started playing with this on CircuitPython 9.2.0 on a Raspberry Pi Pico W and will add notes here.

• I copied crate.py into /lib on the device.
• I tried out importing the other libraries that crate.py needs: requests and base64 and found that neither is installed by default.
• Install Adafruit's requests library onto the device with circup install adafruit-circuitpython-requests -- then import adafruit_requests works in the REPL on the device.
• Install a base64 implementation onto the device with circup install circuitpython-base64 -- then import circuitpython_base64 works in the REPL on the device.

[simonprickett]

Having looked at this, I think we'll need a separate repo for circuitpython-cratedb as the networking and requests are sufficiently different. For example we have to pass networking information into the requests calls, see here.

[amotl]

Thanks for your investigations and for sharing them. First of all, bummer, but on the other hand, c'est la vie, and somehow expected, still unfortunate. However, the library's use only differs on the initialization phase, right?

I think we'll need a separate repo.

I hear you. Let us still think about and evaluate a few options around that? Maybe together tomorrow?

[simonprickett]

I think it'll be a bit messier and be poorly named to try and get 2 for one here. I'll have a go at adapting the current code for CircuitPython and see how it looks. I can also ask someone I know at Adafruit for guidance.

[simonprickett]

I spent a little time looking at other existing CircuitPython libraries that use networked resources. These generally pass in a connection pool and an SSL context, so do work differently than MicroPython and with CircuitPython specific libraries. Here's an MQTT example.

pool = adafruit_connection_manager.get_radio_socketpool(esp)
ssl_context = adafruit_connection_manager.get_radio_ssl_context(esp)

# Set up a MiniMQTT Client
mqtt_client = MQTT.MQTT(
    broker="io.adafruit.com",
    username=aio_username,
    password=aio_key,
    socket_pool=pool,
    ssl_context=ssl_context,
)

NTP seems to adopt a similar approach.

The best approach looks like the creation of a separate library circuitpython-cratedb.

[amotl]

The best approach looks like the creation of a separate library circuitpython-cratedb.

Hi. I don't get it yet, because apparently it would just be about a different interface, right? We could still host it within the same package / repository, in order to be able to reuse shared fragments amongst both library variants?

Background: I am taking a strong position here, biased towards monorepo's where applicable, to keep the number of repositories-to-be-maintained down. If you think my concerns are not applicable, please ignore my admonitions.

[simonprickett]

With a breaking change (now's as good a time as any to do this if we're going to)... I got this to work on CircuitPython.

First, quick change to the imports in cratedb.py... remove everything to do with requests, and add this to account for platform differences in where the base64 encoder is:

try:
    # MicroPython
    from base64 import b64encode
except ImportError:
    # CircuitPython
    from circuitpython_base64 import b64encode

Then, change the constructor in cratedb.py:

class CrateDB:
    def __init__(
        self, requests, host, port=4200, user=None, password=None, schema="doc", use_ssl=True
    ):
        self.requests = requests
        self.user = user
        self.password = password
        self.schema = schema
        self.host = host
        self.port = port
        ...

and change our one usage of requests to use the version passed in:

response = self.requests.post(request_url, headers=headers, json=payload)

Then, usage looks like this on CircuitPython and works fine:

import adafruit_connection_manager
import adafruit_requests

import os
import sys
import time
import wifi
import cratedb

socket_pool = adafruit_connection_manager.get_radio_socketpool(wifi.radio)
ssl_context = adafruit_connection_manager.get_radio_ssl_context(wifi.radio)
requests = adafruit_requests.Session(socket_pool, ssl_context)

crate = cratedb.CrateDB(
    requests=requests, # This is different from the current approach.
    host="<redacted>", # We can advise these go in the TOML file used to store env vars in CircuitPython.
    user="<redacted>", 
    password="<redacted>"
)

# Set up the wifi network and connect.
wifi.radio.connect(os.getenv('CIRCUITPY_WIFI_SSID'), os.getenv('CIRCUITPY_WIFI_PASSWORD'))

ip_addr = wifi.radio.ipv4_address
print(f"Connected to wifi as {ip_addr}")

# Create a table in CrateDB if necessary.
try:
    crate.execute(
        """
            CREATE TABLE IF NOT EXISTS picow_test (
            id TEXT, 
            ts TIMESTAMP WITH TIME ZONE GENERATED ALWAYS AS current_timestamp, 
            temp DOUBLE PRECISION
            )
        """,
        return_response=False,
    )

except Exception as e:
    print("Error creating table:")
    print(e)
    sys.exit(1)

while True:
    # TODO calculate this properly...
    temperature = 22.1

    response = crate.execute(
        "INSERT INTO picow_test (id, temp) VALUES (?, ?)",
        [
            f"{ip_addr}",
            temperature
        ]
    )

    if response["rowcount"] == 1:
        print("Inserted record into CrateDB.")
    
    time.sleep(10)

[simonprickett]

For installation with dependencies on CircuitPython we'll want to look at circup as well as listing dependencies in the README -- see Adafruit's requests for example..

Dependencies we'll need:

• circuitpython-base64
• adafruit-circuitpython-requests

[amotl]

Excellent Simon, thanks for the updates. Adafruit even provides a thoroughly encapsulated GitHub Actions Workflow workflows-circuitpython-libs (see example at circuitpython-requests). It would be cool if your package, even in its hybrid form of supporting both MicroPython and CircuitPython, could adhere to this specification, so builds will succeed when slapping that workflow recipe onto this repository.