Flask-SQLAlchemy 3.x will cause errors in unit testing

[jinmiaoluo]

Latest Flask-SQLAlchemy will cause an error when running make test. The error message is as follows:

self = <SQLAlchemy sqlite:////home/jinmiaoluo/repo/arch-security-tracker/tracker.db>, name = 'create_scoped_session'                                                                                                                                                                                                                                  def __getattr__(self, name: str) -> t.Any:                                                                                                                           
        if name == "db":                                                            
            import warnings                                                         
                           
            warnings.warn(                                                          
                "The 'db' attribute is deprecated and will be removed in"                                                                                                
                " Flask-SQLAlchemy 3.1. The extension is registered directly as"                                                                                         
                " 'app.extensions[\"sqlalchemy\"]'.",                           
                DeprecationWarning,                                                 
                stacklevel=2,      
            )                
            return self
                       
        if name == "relation":
            return self._relation
                                 
        if name == "event":
            return sa.event
                           
        if name.startswith("_"):
            raise AttributeError(name)
                                      
        for mod in (sa, sa.orm):
            if hasattr(mod, name):
                return getattr(mod, name)
     
>       raise AttributeError(name)
E       AttributeError: create_scoped_session

.venv/lib/python3.11/site-packages/flask_sqlalchemy/extension.py:1005: AttributeError

Rolling back Flask-SQLAlchemy to 2.5.x allows unit tests to run smoothly.

diff --git a/requirements.txt b/requirements.txt
index 9519529..8c81793 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -1,6 +1,6 @@
 Flask
 Flask-Login
-Flask-SQLAlchemy
+Flask-SQLAlchemy~=2.5.0
 Flask-Migrate
 Flask-WTF
 flask-talisman

[jelly]

Looking at upgrading now that we have sqlalchemy 2 in Arch Linux, https://github.com/pallets-eco/flask-sqlalchemy/blob/9ecc1d1b835d9a971dd2cb5b01f59cd2d2bca8e5/src/flask_sqlalchemy/extension.py#L389

So create_scoped_session was always internal @anthraxx so we should switch _make_scoped_session but that leaves us with:

FAILED test/test_admin.py::test_delete_last_admin_fails - sqlalchemy.exc.IntegrityError: (sqlite3.IntegrityError) UNIQUE constraint failed: user.email
FAILED test/test_admin.py::test_edit_requires_admin - sqlalchemy.exc.IntegrityError: (sqlite3.IntegrityError) UNIQUE constraint failed: user.name
ERROR test/test_admin.py::test_delete_user - sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) database is locked
ERROR test/test_admin.py::test_deactive_user - sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) database is locked
ERROR test/test_login.py::test_logout - sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) database is locked

The issues are not reproducible when running the single test:

make test PYTEST_OPTIONS="-vs -k test_delete_last_admin_fails"  PYTEST_INPUT=test/test_admin.py PYTEST_COVERAGE_OPTIONS=

But are when running all test_admin tests, so the issue is related to either the code or _scoped_session
Changelog: https://flask-sqlalchemy.palletsprojects.com/en/3.1.x/changes/#version-3-1-2

[jelly]

I suppose the issue is related to:

https://github.com/pallets-eco/flask-sqlalchemy/blob/9ecc1d1b835d9a971dd2cb5b01f59cd2d2bca8e5/docs/contexts.rst#L5

[jelly]

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <sqlalchemy.dialects.sqlite.pysqlite.SQLiteDialect_pysqlite object at 0x7481baa89550>, cursor = <sqlite3.Cursor object at 0x7481b9ae3bc0>, statement = '\nDROP TABLE advisory', parameters = (), context = <sqlalchemy.dialects.sqlite.base.SQLiteExecutionContext object at 0x7481b9567110>

    def do_execute(self, cursor, statement, parameters, context=None):
>       cursor.execute(statement, parameters)
E       sqlite3.OperationalError: database is locked

.venv/lib/python3.12/site-packages/sqlalchemy/engine/default.py:736: OperationalError

The above exception was the direct cause of the following exception:

app = <Flask 'tracker'>, db = <SQLAlchemy sqlite:////home/jelle/projects/arch-security-tracker/tracker.db>, client = <FlaskClient <Flask 'tracker'>>, request = <SubRequest 'run_scoped' for <Function test_deactive_user>>

    @pytest.fixture(autouse=True, scope='function')
    def run_scoped(app, db, client, request):
        with app.app_context():
            connection = db.engine.connect()
            transaction = connection.begin()

            options = dict(bind=connection, binds={})
            session = db._make_scoped_session(options=options)

            db.session = session
            db.create_all()

            with client:
                yield

>           db.drop_all()

test/conftest.py:78:

[jelly]

My unsuccesful patch:

diff --git a/test/conftest.py b/test/conftest.py
index 3419d85..78f5bcb 100644
--- a/test/conftest.py
+++ b/test/conftest.py
@@ -63,22 +63,12 @@ def db(app, request):
 @pytest.fixture(autouse=True, scope='function')
 def run_scoped(app, db, client, request):
     with app.app_context():
-        connection = db.engine.connect()
-        transaction = connection.begin()
-
-        options = dict(bind=connection, binds={})
-        session = db.create_scoped_session(options=options)
-
-        db.session = session
         db.create_all()
 
         with client:
             yield
 
         db.drop_all()
-        transaction.rollback()
-        connection.close()
-        session.remove()
 
 
 @pytest.fixture(scope='function')