[Python] Worker Initialized but Not Processing Jobs

[akramhecini]

I have implemented a Flask API with a BullMQ Worker to handle report generation. Although the job is successfully added to the queue and the worker appears to be initialized correctly ("Worker is ready and listening for jobs." is logged), the report generation function is never executed.

Relevant Code
The key parts of the code are as follows:

• Adding the job to the queue in the Flask API:

@app.route('/generate-report', methods=['POST'])
async def generate_report():
    data = request.get_json()
    survey_id = data.get("id")

    if not survey_id:
        return jsonify({"error": "ID missing"}), 400
    job = await queue.add("generate_report", {"survey_id": survey_id})
    logger.info(f"Added job to queue with ID: {job.id}")

    return jsonify({"message": "Task in progress", "task_id": job.id})

• Job processing by the Worker:

async def process_job(job):
    survey_id = job.data.get("survey_id")
    logger.info(f"Processing job with survey ID: {survey_id}")
    return await generate_report_task(survey_id)

• Worker Initialization:

async def main_worker():
    try:
        logger.info("Initializing worker...")
        worker = Worker("generate_reports_queue", process_job)
        logger.info("Worker is ready and listening for jobs.")
        await asyncio.Future()  # Keep the worker running
    except Exception as e:
        logger.error(f"Worker error: {e}", exc_info=True)

Expected Behavior
I expected the generate_report_task function to be called and the report to be generated once the job is picked up by the worker.

Observed Behavior

• The job is successfully added to the queue.
• The message "Worker is ready and listening for jobs." appears in the logs.
• However, the generate_report_task function is never called.

Technical Details

• Python 3.11
• Running on WSL2

Request for Help
I believe the worker is not correctly intercepting jobs from the queue. Any suggestions or troubleshooting tips would be greatly appreciated. Thank you in advance for your help!

[manast]

Where is the code that instantiates the queue instance?