4.0a0

• Upsert operations now use SQLite's INSERT ... ON CONFLICT SET syntax on all SQLite versions later than 3.23.1. This is a very slight breaking change for apps that depend on the previous INSERT OR IGNORE followed by UPDATE behavior. (#652)
• Python library users can opt-in to the previous implementation by passing use_old_upsert=True to the Database() constructor, see Alternative upserts using INSERT OR IGNORE.
• Dropped support for Python 3.8, added support for Python 3.13. (#646)
• sqlite-utils tui is now provided by the sqlite-utils-tui plugin. (#648)
• Test suite now also runs against SQLite 3.23.1, the last version (from 2018-04-10) before the new INSERT ... ON CONFLICT SET syntax was added. (#654)

3.38

• Plugins can now reuse the implementation of the sqlite-utils memory CLI command with the new return_db=True parameter. (#643)
• table.transform() now recreates indexes after transforming a table. A new sqlite_utils.db.TransformError exception is raised if these indexes cannot be recreated due to conflicting changes to the table such as a column rename. Thanks, Mat Miller. (#633)
• table.search() now accepts a include_rank=True parameter, causing the resulting rows to have a rank column showing the calculated relevance score. Thanks, liunux4odoo. (#628)
• Fixed an error that occurred when creating a strict table with at least one floating point column. These FLOAT columns are now correctly created as REAL as well, but only for strict tables. (#644)

3.38a0

• Plugins can now reuse the sqlite-utils memory command with the new return_db=True parameter. #643

3.37

• The create-table and insert-files commands all now accept multiple --pk options for compound primary keys. (#620)
• Now tested against Python 3.13 pre-release. (#619)
• Fixed a crash that can occur in environments with a broken numpy installation, producing a module 'numpy' has no attribute 'int8'. (#632)

3.36

• Support for creating tables in SQLite STRICT mode. Thanks, Taj Khattra. (#344)
  • CLI commands create-table, insert and upsert all now accept a --strict option.
  • Python methods that can create a table - table.create() and insert/upsert/insert_all/upsert_all all now accept an optional strict=True parameter.
  • The transform command and table.transform() method preserve strict mode when transforming a table.
• The sqlite-utils create-table command now accepts str, int and bytes as aliases for text, integer and blob respectively. (#606)

3.35.2

• The --load-extension=spatialite option and find_spatialite() utility function now both work correctly on arm64 Linux. Thanks, Mike Coats. (#599)
• Fix for bug where sqlite-utils insert could cause your terminal cursor to disappear. Thanks, Luke Plant. (#433)
• datetime.timedelta values are now stored as TEXT columns. Thanks, Harald Nezbeda. (#522)
• Test suite is now also run against Python 3.12.

3.35.1

• Fixed a bug where table.transform() would sometimes re-assign the rowid values for a table rather than keeping them consistent across the operation. (#592)

3.35

Adding foreign keys to a table no longer uses PRAGMA writable_schema = 1 to directly manipulate the sqlite_master table. This was resulting in errors in some Python installations where the SQLite library was compiled in a way that prevented this from working, in particular on macOS. Foreign keys are now added using the table transformation mechanism instead. (#577)

This new mechanism creates a full copy of the table, so it is likely to be significantly slower for large tables, but will no longer trigger table sqlite_master may not be modified errors on platforms that do not support PRAGMA writable_schema = 1.

A new plugin, sqlite-utils-fast-fks, is now available for developers who still want to use that faster but riskier implementation.

Other changes:

• The table.transform() method has two new parameters: foreign_keys= allows you to replace the foreign key constraints defined on a table, and add_foreign_keys= lets you specify new foreign keys to add. These complement the existing drop_foreign_keys= parameter. (#577)
• The sqlite-utils transform command has a new --add-foreign-key option which can be called multiple times to add foreign keys to a table that is being transformed. (#585)
• sqlite-utils convert now has a --pdb option for opening a debugger on the first encountered error in your conversion script. (#581)
• Fixed a bug where sqlite-utils install -e '.[test]' option did not work correctly.

3.34

This release introduces a new plugin system. (#567)

• Documentation describing how to build a plugin.
• Plugin hook: register_commands(cli), for plugins to add extra commands to sqlite-utils. (#569)
• Plugin hook: prepare_connection(conn). Plugins can use this to help prepare the SQLite connection to do things like registering custom SQL functions. Thanks, Alex Garcia. (#574)
• sqlite_utils.Database(..., execute_plugins=False) option for disabling plugin execution. (#575)
• sqlite-utils install -e path-to-directory option for installing editable code. This option is useful during the development of a plugin. (#570)
• table.create(...) method now accepts replace=True to drop and replace an existing table with the same name, or ignore=True to silently do nothing if a table already exists with the same name. (#568)
• sqlite-utils insert ... --stop-after 10 option for stopping the insert after a specified number of records. Works for the upsert command as well. (#561)
• The --csv and --tsv modes for insert now accept a --empty-null option, which cases empty strings in the CSV file to be stored as null in the database. (#563)
• New db.rename_table(table_name, new_name) method for renaming tables. (#565)
• sqlite-utils rename-table my.db table_name new_name command for renaming tables. (#565)
• The table.transform(...) method now takes an optional keep_table=new_table_name parameter, which will cause the original table to be renamed to new_table_name rather than being dropped at the end of the transformation. (#571)
• Documentation now notes that calling table.transform() without any arguments will reformat the SQL schema stored by SQLite to be more aesthetically pleasing. (#564)

3.33

• sqlite-utils will now use sqlean.py in place of sqlite3 if it is installed in the same virtual environment. This is useful for Python environments with either an outdated version of SQLite or with restrictions on SQLite such as disabled extension loading or restrictions resulting in the sqlite3.OperationalError: table sqlite_master may not be modified error. (#559)
• New with db.ensure_autocommit_off() context manager, which ensures that the database is in autocommit mode for the duration of a block of code. This is used by db.enable_wal() and db.disable_wal() to ensure they work correctly with pysqlite3 and sqlean.py.
• New db.iterdump() method, providing an iterator over SQL strings representing a dump of the database. This uses sqlite-dump if it is available, otherwise falling back on the conn.iterdump() method from sqlite3. Both pysqlite3 and sqlean.py omit support for iterdump() - this method helps paper over that difference.