Everytime I run db:migrate, db/queue_schema.rb is emptied, and tables don't exist (rails 8.0.3)

[rathboma]

Hey team,

I started using solid_queue in development, but I'm continually running into a problem moving between computers.

Whenever I move from my laptop to my desktop and run db:migrate, I seem to be missing one solid_queue table or another, and the db/queue_schema.rb is emptied.

Example error from this morning:

/home/rathboma/.rvm/gems/ruby-3.2.3/gems/activerecord-8.0.2/lib/active_record/connection_adapters/postgresql/database_statements.rb:160:in `exec': ERROR:  relation "solid_queue_recurring_tasks" does not exist (PG::UndefinedTable)
LINE 10:  WHERE a.attrelid = '"solid_queue_recurring_tasks"'::regclas...

I've seen many issues that supposedly deal with this, but none of them seem to work for me. I really don't know why it is behaving this way -- I would expect it to have migration files, but it doesn't seem to?

[rosa]

Hey @rathboma, you don't have to run db:migrate. You just need to run db:prepare, as the instructions explain.

[rathboma]

Hey Rosa! If I run db:prepare, I get the same error, although the db/queue_schema.rb isn't deleted.

[rosa]

Hmm... that's weird! Have you followed the instructions to set up solid queue in development? 🤔

[rathboma]

Yes!

I had it enabled for production, then updated my code to add it in for development last week, now I can get it working by wiping my database, restoring the queue_schema file, then running prepare, but otherwise it refuses to set it up right.

My code looks like this:

development:
  primary:
    adapter: postgresql
    database: fuego_development
    pool: 5
    timeout: 5000
    host: localhost
    username: postgres
    port: 5556
    password: example
  queue:
    adapter: postgresql
    database: fuego_development_queue
    migrations_paths: db/queue_migrate
    pool: 5
    timeout: 5000
    host: localhost
    username: postgres
    port: 5556
    password: example

# development.rb
  config.active_job.queue_adapter = :solid_queue
  config.solid_queue.connects_to = { database: { writing: :queue } }
  config.solid_queue.logger = ActiveSupport::Logger.new(STDOUT)

There's nothing in the queue database:

[rosa]

Hey @rathboma, no, that's correct. The queue_schema.rb file is enough to set up the DB. The schema_migrations table is expected to be empty.

Could you check the contents of your queue_schema.rb file?

[rathboma]

Did some debugging, seems maybe this is because I ended up with a blank queue_schema.rb, then ran db:prepare which created an empty fuego_development_queue database, then anytime I ran prepare or migrate it didn't do anything.

Can you explain the migrate vs prepare thing? Can I not run db:migrate anymore for fear of it wiping my queue_schema?

[rosa]

No, db:migrate shouldn't touch queue_schema.rb, that seems like a bug in Rails. If you restore the queue_schema.rb file and run db:prepare, does it work fine?

[rathboma]

So the problem was:

1. The development_queue database EXISTED
2. ...but it was empty

I must have run a stray command somewhere to do this.

This makes db:prepare do nothing (oh the database exists, I'm all good)
This makes db:migrate wipe out the queue_schema.rb - I guess because it looks in the database, and there's nothing there, so rebuilds the queue_schema.rb to match. -- Maybe this is the bit that it shouldn't do?

Now when I drop the queue database, both migrate and prepare work correctly.

[ericdfields]

I ran into the same issue. In my case, I merged in some code from another branch where there was a migration. I cleaned up the conflicts and ran bin/rails db:migrate:reset. Then ran into the same solid queue issues outlined above. So far, dropping the development_queue branch seems to get me moving forward.