Cache connection established properly, but no keys found inside Redis

[BartoszOrnoch]

Hello,

I'm deploying superset on kubernetes using helmchart. Everything seems to be working fine. Wait-for-redis init container starts. Queries are saved in backend:

requests to api are being made:
http://dummy.disscussion.pl/api/v1/sqllab/results/?q=(key:%2779e4dfbb-63a9-468b-a10a-7df129278ac1%27,rows:10000)
and logs are saying, that connection to redis is set:

Loaded your LOCAL configuration at [/app/superset_config.py]
- ** ---------- .> transport:   redis://default:**@redis.myredis.pl:6544/0
- ** ---------- .> results:     redis://default:**@redis.myredis.pl:6544/0
- *** --- * --- .> concurrency: 48 (prefork)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)

However I'm not seeing any keys in my redis and I'm under impresion that superset is using in-memory cache.

redis.myredis.pl:6544> SELECT 0;
(error) ERR value is not an integer or out of range
redis.myredis.pl:6544> KEYS *;
(empty array)

Below is my superset_config.py redis configuration.

REDIS_PASSWORD = get_env_variable("REDIS_PASSWORD")
REDIS_HOST = 'redis.myredis.pl' #changed for discussion case
REDIS_PORT = 6544

encoded_password = quote_plus(REDIS_PASSWORD) #Password is qouted because of "/" sign which causes error

# Cache configuration
DATA_CACHE_CONFIG = {
    "CACHE_TYPE": "RedisCache",
    "CACHE_DEFAULT_TIMEOUT": 600,
    "CACHE_KEY_PREFIX": "superset_",
    "CACHE_REDIS_URL": f"redis://default:{encoded_password}@{REDIS_HOST}:{REDIS_PORT}/1",
}

# Filter state cache configuration
FILTER_STATE_CACHE_CONFIG = {
    "CACHE_TYPE": "RedisCache",
    "CACHE_DEFAULT_TIMEOUT": 86400,
    "CACHE_KEY_PREFIX": "superset_filter_",
    "CACHE_REDIS_URL": f"redis://default:{encoded_password}@{REDIS_HOST}:{REDIS_PORT}/2",
}

EXPLORE_FORM_DATA_CACHE_CONFIG = {
    "CACHE_TYPE": "RedisCache",
    "CACHE_DEFAULT_TIMEOUT": 86400,
    "CACHE_KEY_PREFIX": "superset_explore_",
    "CACHE_REDIS_URL": f"redis://default:{encoded_password}@{REDIS_HOST}:{REDIS_PORT}/3",
}

RATELIMIT_STORAGE_URI = f"redis://default:{encoded_password}@{REDIS_HOST}:{REDIS_PORT}/4"

CACHE_CONFIG = DATA_CACHE_CONFIG

# Celery configuration class
class CeleryConfig(object):
    broker_url = f"redis://default:{encoded_password}@{REDIS_HOST}:{REDIS_PORT}/0"
    result_backend = f"redis://default:{encoded_password}@{REDIS_HOST}:{REDIS_PORT}/0"
    imports = (
        "superset.sql_lab",
        "superset.tasks.scheduler",
    )
    worker_prefetch_multiplier = 10
    task_acks_late = True
    task_annotations = {
        "sql_lab.get_sql_results": {
            "rate_limit": "100/s",
        },
    }
    beat_schedule = {
        "reports.scheduler": {
            "task": "reports.scheduler",
            "schedule": crontab(minute="*", hour="*"),
        },
        "reports.prune_log": {
            "task": "reports.prune_log",
            "schedule": crontab(minute=10, hour=0),
        },
    }


RESULTS_BACKEND = RedisCache(host=REDIS_HOST, port=REDIS_PORT, password=REDIS_PASSWORD, key_prefix='superset_results')
# Assign Celery configuration
CELERY_CONFIG = CeleryConfig
CELERY_BROKER_CONNECTION_RETRY_ON_STARTUP = True

Is everything configured correctly? Maybe something is missing?

[dosubot[bot]]

Hey @BartoszOrnoch! I'm here to help you out with any bugs, questions, or contributions. Let's squash this bug together! 🤖

It seems like your Redis configuration is mostly set up correctly, but there might be a misunderstanding regarding which Redis databases are being used for different purposes. Based on your configuration, Superset is using multiple Redis databases for different caching and task management purposes:

1. Data Cache: Database 1 is used for the data cache (DATA_CACHE_CONFIG).
2. Filter State Cache: Database 2 is used for the filter state cache (FILTER_STATE_CACHE_CONFIG).
3. Explore Form Data Cache: Database 3 is used for the explore form data cache (EXPLORE_FORM_DATA_CACHE_CONFIG).
4. Rate Limit Storage: Database 4 is used for rate limit storage (RATELIMIT_STORAGE_URI).
5. Celery Broker and Result Backend: Database 0 is used for both the Celery broker and result backend (CeleryConfig).

If you are not seeing any keys in Redis, make sure you are checking the correct database. You can switch to the appropriate database using the SELECT command in the Redis CLI. For example, to check the data cache, you should select database 1:

redis.myredis.pl:6544> SELECT 1
OK
redis.myredis.pl:6544[1]> KEYS *

Ensure that you are selecting the correct database number that corresponds to the cache or backend you want to inspect. If you still don't see any keys, verify that your application is correctly writing to Redis and that there are no errors in the logs indicating issues with Redis connectivity or operations.

To continue talking to Dosu, mention @dosu.

[BartoszOrnoch]

Okey, the problem was ';' sign at the end of my redis-cli commands xD. Thanks for help mr Dosubot