Climsoft data models

[faysal-ishtiaq]

defined composite primary keys using this sql script located at opencdms-test-data/schemas/climsoft/v4/v4.1.1/primary_key_definition.sql in my local copy of opencdms-test-data repo

ALTER TABLE mariadb_climsoft_db_v4.observationfinal ADD PRIMARY KEY (recordedFrom, describedBy, obsDatetime, obsLevel);
ALTER TABLE mariadb_climsoft_db_v4.observationinitial ADD PRIMARY KEY (recordedFrom, describedBy, obsDatetime, qcStatus,acquisitionType, obsLevel);
ALTER TABLE mariadb_climsoft_db_v4.instrumentinspection ADD PRIMARY KEY (performedOn, inspectionDatetime );
ALTER TABLE mariadb_climsoft_db_v4.faultresolution ADD PRIMARY KEY (resolvedDatetime, associatedWith );
ALTER TABLE mariadb_climsoft_db_v4.stationelement ADD PRIMARY KEY (recordedFrom, describedBy, recordedWith, beginDate);
ALTER TABLE mariadb_climsoft_db_v4.stationlocationhistory ADD PRIMARY KEY (belongsTo, openingDatetime);
ALTER TABLE mariadb_climsoft_db_v4.stationqualifier ADD PRIMARY KEY (Qualifier, qualifierBeginDate, qualifierEndDate,belongsTo);
ALTER TABLE mariadb_climsoft_db_v4.observationschedule ADD PRIMARY KEY (classifiedInto, startTime, endTime);
ALTER TABLE mariadb_climsoft_db_v4.physicalfeature ADD PRIMARY KEY (associatedWith, beginDate, classifiedInto, description);
ALTER TABLE mariadb_climsoft_db_v4.featuregeographicalposition ADD PRIMARY KEY (belongsTo, observedOn);
ALTER TABLE mariadb_climsoft_db_v4.paperarchive ADD PRIMARY KEY (belongsTo, formDatetime, classifiedInto );

Then executed following commands:

to create tables

 mysql -h localhost -P 3306 --protocol tcp -u root -p < ./opencdms-test-data/schemas/climsoft/v4/v4.1.1/climsoft_v4_all.sql 

to create primary keys

mysql -h localhost -P 3306 --protocol tcp -u root -p < ./opencdms-test-data/schemas/climsoft/v4/v4.1.1/primary_key_definition.sql

to generate models

sqlacodegen mysql+pymysql://root:root@localhost/mariadb_climsoft_db_v4 > ./opencdms/models/climsoft/v4_1_1.py

But there are a lot of other models which are not listed here https://github.com/opencdms/pyopencdms/files/6995106/Climsoft.Data.Model.Tables.and.Keys.pdf

[faysal-ishtiaq]

#2

[isedwards]

@faysal-ishtiaq - Ignore my previous comment about detecting which tables are missing primary keys for now, we can pick that up later in #15 (feel free to add comments to #15 if you started working on this).

For now, I've been through the Climsoft DDL manually (climsoft/climsoft-database#2 (comment)) and we can safely add the following as primary keys:

Table name	Candidate Primary Key
aws_lsi	PRIMARY KEY (`Cols`)
aws_lsi1	PRIMARY KEY (`Cols`)
aws_malawi12	PRIMARY KEY (`No`)
aws_rema1	PRIMARY KEY (`Cols`)
aws_rwanda_rain	PRIMARY KEY (`Cols`)
qcabslimits	PRIMARY KEY (`StationId`, `ElementId`, `Datetime`)
routinereporttransmission	PRIMARY KEY (`reportClass`, `reportDatetime`, `reportedFrom`)
stationidalias	PRIMARY KEY (`idAlias`)

In your primary_key_definition.sql - could you add these to the end, run the script against the database and then rerun sqlacodegen?

Given the SQLAlchemy models you've created in opencdms/models/climsoft/v4_1_1.py, the final tasks in #2 are:

1. Run get_schema_diff against the original Climsoft schema (you'll need to drop the database and then restore the original mariadb_climsoft_db_v4).
2. Drop again and restore mariadb_climsoft_test_db_v4. Run get_schema_diff against this version and see whether the same differences are reported

Could you paste the output of get_schema_diff in a comment below and we'll decide how to deal with the differences?

[faysal-ishtiaq]

@isedwards Okay. I will update you when done.

[faysal-ishtiaq]

I followed your instruction and added the new primary keys to db. And now opencdms-test-data/schemas/climsoft/v4/v4.1.1/primary_key_definition.sql looks like:

ALTER TABLE mariadb_climsoft_db_v4.observationfinal ADD PRIMARY KEY (recordedFrom, describedBy, obsDatetime, obsLevel);
ALTER TABLE mariadb_climsoft_db_v4.observationinitial ADD PRIMARY KEY (recordedFrom, describedBy, obsDatetime, qcStatus,acquisitionType, obsLevel);
ALTER TABLE mariadb_climsoft_db_v4.instrumentinspection ADD PRIMARY KEY (performedOn, inspectionDatetime );
ALTER TABLE mariadb_climsoft_db_v4.faultresolution ADD PRIMARY KEY (resolvedDatetime, associatedWith );
ALTER TABLE mariadb_climsoft_db_v4.stationelement ADD PRIMARY KEY (recordedFrom, describedBy, recordedWith, beginDate);
ALTER TABLE mariadb_climsoft_db_v4.stationlocationhistory ADD PRIMARY KEY (belongsTo, openingDatetime);
ALTER TABLE mariadb_climsoft_db_v4.stationqualifier ADD PRIMARY KEY (Qualifier, qualifierBeginDate, qualifierEndDate,belongsTo);
ALTER TABLE mariadb_climsoft_db_v4.observationschedule ADD PRIMARY KEY (classifiedInto, startTime, endTime);
ALTER TABLE mariadb_climsoft_db_v4.physicalfeature ADD PRIMARY KEY (associatedWith, beginDate, classifiedInto, description);
ALTER TABLE mariadb_climsoft_db_v4.featuregeographicalposition ADD PRIMARY KEY (belongsTo, observedOn);
ALTER TABLE mariadb_climsoft_db_v4.paperarchive ADD PRIMARY KEY (belongsTo, formDatetime, classifiedInto );
ALTER TABLE mariadb_climsoft_db_v4.aws_lsi ADD PRIMARY KEY (Cols);
ALTER TABLE mariadb_climsoft_db_v4.aws_lsi1 ADD PRIMARY KEY (Cols);
ALTER TABLE mariadb_climsoft_db_v4.aws_malawi12 ADD PRIMARY KEY (No);
ALTER TABLE mariadb_climsoft_db_v4.aws_rema1 ADD PRIMARY KEY (Cols);
ALTER TABLE mariadb_climsoft_db_v4.aws_rwanda_rain ADD PRIMARY KEY (Cols);
ALTER TABLE mariadb_climsoft_db_v4.qcabslimits ADD PRIMARY KEY (StationId, ElementId, Datetime);
ALTER TABLE mariadb_climsoft_db_v4.routinereporttransmission ADD PRIMARY KEY (reportClass, reportDatetime, reportedFrom);
ALTER TABLE mariadb_climsoft_db_v4.stationidalias ADD PRIMARY KEY (idAlias);

Then I dropped the databases and imported the original DDL again.

mysql -h localhost -P 3306 --protocol tcp -u root -p < ./opencdms-test-data/schemas/climsoft/v4/v4.1.1/climsoft_v4_all.sql

Then added the primary keys

mysql -h localhost -P 3306 --protocol tcp -u root -p < ./opencdms-test-data/schemas/climsoft/v4/v4.1.1/primary_key_definition.sql

Then generated the models

 sqlacodegen mysql+pymysql://root:root@localhost/mariadb_climsoft_db_v4 > ./opencdms/models/climsoft/v4_1_1.py

Then dropped the database again and then imported the DDL file. Then ran the following code to get the schema diff result:

from opencdms.models import get_schema_diff
from opencdms.models.climsoft.v4_1_1 import metadata


test_db_schema_diff = get_schema_diff(metadata, 'mysql+pymysql://root:root@localhost/mariadb_climsoft_test_db_v4')
main_db_schema_diff = get_schema_diff(metadata, 'mysql+pymysql://root:root@localhost/mariadb_climsoft_db_v4')

len(test_db_schema_diff) == len(main_db_schema_diff) # returns False

[isedwards]

Could you paste the contents of both test_db_schema_diff and main_db_schema_diff?

[faysal-ishtiaq]

main_db_schema_diff

[('remove_index', Index('Identification', Column('Element_Name', VARCHAR(length=50), table=<aws_malawi1>))), ('add_index', Index('ix_aws_malawi1_Element_Name', Column('Element_Name', String(length=50), table=<aws_malawi1>))), ('remove_index', Index('identification', Column('Element_Name', VARCHAR(length=50), table=<aws_rwanda1>))), ('add_index', Index('ix_aws_rwanda1_Element_Name', Column('Element_Name', String(length=50), table=<aws_rwanda1>))), [('modify_nullable', None, 'faultresolution', 'resolvedDatetime', {'existing_type': VARCHAR(length=50), 'existing_server_default': False, 'existing_comment': None}, True, False)], [('modify_nullable', None, 'faultresolution', 'associatedWith', {'existing_type': BIGINT(), 'existing_server_default': False, 'existing_comment': None}, True, False)], ('add_index', Index('ix_faultresolution_associatedWith', Column('associatedWith', BigInteger(), table=<faultresolution>, primary_key=True, nullable=False))), [('modify_nullable', None, 'featuregeographicalposition', 'observedOn', {'existing_type': VARCHAR(length=50), 'existing_server_default': False, 'existing_comment': None}, True, False)], ('remove_index', Index('code', Column('instrumentId', VARCHAR(length=255), table=<instrument>, primary_key=True, nullable=False))), ('add_index', Index('ix_instrument_installedAt', Column('installedAt', String(length=255), table=<instrument>))), ('add_index', Index('ix_instrument_instrumentId', Column('instrumentId', String(length=255), table=<instrument>, primary_key=True, nullable=False))), ('remove_index', Index('report_id', Column('reportId', BIGINT(), table=<instrumentfaultreport>, primary_key=True, nullable=False))), ('add_index', Index('ix_instrumentfaultreport_reportId', Column('reportId', BigInteger(), table=<instrumentfaultreport>, primary_key=True, nullable=False))), ('add_index', Index('ix_instrumentfaultreport_reportedFrom', Column('reportedFrom', String(length=255), table=<instrumentfaultreport>))), [('modify_nullable', None, 'instrumentinspection', 'performedOn', {'existing_type': VARCHAR(length=255), 'existing_server_default': False, 'existing_comment': None}, True, False)], [('modify_nullable', None, 'instrumentinspection', 'inspectionDatetime', {'existing_type': VARCHAR(length=50), 'existing_server_default': False, 'existing_comment': None}, True, False)], ('add_index', Index('ix_instrumentinspection_performedAt', Column('performedAt', String(length=255), table=<instrumentinspection>))), ('remove_index', Index('elementCode', Column('elementId', BIGINT(), table=<obselement>, primary_key=True, nullable=False, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x7f72155bcbe0>, for_update=False)))), ('add_index', Index('ix_obselement_elementId', Column('elementId', BigInteger(), table=<obselement>, primary_key=True, nullable=False, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x7f7216396970>, for_update=False)))), [('modify_nullable', None, 'observationfinal', 'describedBy', {'existing_type': BIGINT(), 'existing_server_default': False, 'existing_comment': None}, True, False)], [('modify_nullable', None, 'observationfinal', 'obsDatetime', {'existing_type': DATETIME(), 'existing_server_default': False, 'existing_comment': None}, True, False)], [('modify_nullable', None, 'observationfinal', 'obsLevel', {'existing_type': VARCHAR(length=255), 'existing_server_default': DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x7f72155ceee0>, for_update=False), 'existing_comment': None}, True, False)], ('remove_index', Index('obsElementObservationInitial', Column('describedBy', BIGINT(), table=<observationfinal>))), ('remove_index', Index('stationObservationInitial', Column('recordedFrom', VARCHAR(length=255), table=<observationfinal>, nullable=False))), ('add_index', Index('ix_observationfinal_describedBy', Column('describedBy', BigInteger(), table=<observationfinal>, primary_key=True, nullable=False))), ('add_index', Index('ix_observationfinal_recordedFrom', Column('recordedFrom', String(length=255), table=<observationfinal>, primary_key=True, nullable=False))), [('modify_nullable', None, 'observationinitial', 'describedBy', {'existing_type': BIGINT(), 'existing_server_default': False, 'existing_comment': None}, True, False)], [('modify_nullable', None, 'observationinitial', 'obsDatetime', {'existing_type': DATETIME(), 'existing_server_default': False, 'existing_comment': None}, True, False)], [('modify_nullable', None, 'observationinitial', 'obsLevel', {'existing_type': VARCHAR(length=255), 'existing_server_default': False, 'existing_comment': None}, True, False)], [('modify_nullable', None, 'observationinitial', 'qcStatus', {'existing_type': INTEGER(), 'existing_server_default': DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x7f72152ae730>, for_update=False), 'existing_comment': None}, True, False)], [('modify_nullable', None, 'observationinitial', 'acquisitionType', {'existing_type': INTEGER(), 'existing_server_default': DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x7f72152ae880>, for_update=False), 'existing_comment': None}, True, False)], ('remove_index', Index('obsElementObservationInitial', Column('describedBy', BIGINT(), table=<observationinitial>))), ('remove_index', Index('stationObservationInitial', Column('recordedFrom', VARCHAR(length=255), table=<observationinitial>, nullable=False))), ('add_index', Index('ix_observationinitial_describedBy', Column('describedBy', BigInteger(), table=<observationinitial>, primary_key=True, nullable=False))), ('add_index', Index('ix_observationinitial_recordedFrom', Column('recordedFrom', String(length=255), table=<observationinitial>, primary_key=True, nullable=False))), [('modify_nullable', None, 'observationschedule', 'classifiedInto', {'existing_type': VARCHAR(length=255), 'existing_server_default': False, 'existing_comment': None}, True, False)], [('modify_nullable', None, 'observationschedule', 'startTime', {'existing_type': VARCHAR(length=50), 'existing_server_default': False, 'existing_comment': None}, True, False)], [('modify_nullable', None, 'observationschedule', 'endTime', {'existing_type': VARCHAR(length=50), 'existing_server_default': False, 'existing_comment': None}, True, False)], ('remove_index', Index('scheduleClassIdeification', Column('scheduleClass', VARCHAR(length=255), table=<obsscheduleclass>, primary_key=True, nullable=False, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x7f72152c8b50>, for_update=False)))), ('add_index', Index('ix_obsscheduleclass_refersTo', Column('refersTo', String(length=255), table=<obsscheduleclass>))), ('add_index', Index('ix_obsscheduleclass_scheduleClass', Column('scheduleClass', String(length=255), table=<obsscheduleclass>, primary_key=True, nullable=False, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x7f7216180bb0>, for_update=False)))), [('modify_nullable', None, 'paperarchive', 'belongsTo', {'existing_type': VARCHAR(length=255), 'existing_server_default': False, 'existing_comment': None}, True, False)], [('modify_nullable', None, 'paperarchive', 'formDatetime', {'existing_type': DATETIME(), 'existing_server_default': False, 'existing_comment': None}, True, False)], [('modify_nullable', None, 'paperarchive', 'classifiedInto', {'existing_type': VARCHAR(length=50), 'existing_server_default': False, 'existing_comment': None}, True, False)], ('add_index', Index('ix_paperarchive_classifiedInto', Column('classifiedInto', String(length=50), table=<paperarchive>, primary_key=True, nullable=False))), ('remove_index', Index('paperarchivedef', Column('formId', VARCHAR(length=50), table=<paperarchivedefinition>, primary_key=True, nullable=False))), ('add_index', Index('ix_paperarchivedefinition_formId', Column('formId', String(length=50), table=<paperarchivedefinition>, primary_key=True, nullable=False))), [('modify_nullable', None, 'physicalfeature', 'beginDate', {'existing_type': VARCHAR(length=50), 'existing_server_default': False, 'existing_comment': None}, True, False)], [('modify_nullable', None, 'physicalfeature', 'description', {'existing_type': VARCHAR(length=255), 'existing_server_default': False, 'existing_comment': None}, True, False)], [('modify_nullable', None, 'physicalfeature', 'classifiedInto', {'existing_type': VARCHAR(length=255), 'existing_server_default': False, 'existing_comment': None}, True, False)], ('remove_index', Index('physicalFeatureidentification_idx', Column('classifiedInto', VARCHAR(length=255), table=<physicalfeature>))), ('remove_index', Index('stationfeature', Column('associatedWith', VARCHAR(length=255), table=<physicalfeature>, nullable=False))), ('add_index', Index('ix_physicalfeature_associatedWith', Column('associatedWith', String(length=255), table=<physicalfeature>, primary_key=True, nullable=False))), ('add_index', Index('ix_physicalfeature_classifiedInto', Column('classifiedInto', String(length=255), table=<physicalfeature>, primary_key=True, nullable=False))), ('remove_index', Index('stationFeatureClass', Column('featureClass', VARCHAR(length=255), table=<physicalfeatureclass>, primary_key=True, nullable=False))), ('add_index', Index('ix_physicalfeatureclass_featureClass', Column('featureClass', String(length=255), table=<physicalfeatureclass>, primary_key=True, nullable=False))), ('add_index', Index('ix_physicalfeatureclass_refersTo', Column('refersTo', String(length=255), table=<physicalfeatureclass>))), [('modify_nullable', None, 'qcabslimits', 'ElementId', {'existing_type': BIGINT(), 'existing_server_default': False, 'existing_comment': None}, True, False)], [('modify_nullable', None, 'qcabslimits', 'Datetime', {'existing_type': DATETIME(), 'existing_server_default': False, 'existing_comment': None}, True, False)], ('remove_index', Index('report_code', Column('reportCode', VARCHAR(length=255), table=<routinereportdefinition>))), ('add_index', Index('ix_routinereportdefinition_reportCode', Column('reportCode', String(length=255), table=<routinereportdefinition>))), [('modify_nullable', None, 'routinereporttransmission', 'reportClass', {'existing_type': VARCHAR(length=255), 'existing_server_default': False, 'existing_comment': None}, True, False)], [('modify_nullable', None, 'routinereporttransmission', 'reportDatetime', {'existing_type': DATETIME(), 'existing_server_default': False, 'existing_comment': None}, True, False)], [('modify_nullable', None, 'routinereporttransmission', 'reportedFrom', {'existing_type': VARCHAR(length=255), 'existing_server_default': False, 'existing_comment': None}, True, False)], ('add_index', Index('ix_routinereporttransmission_reportedFrom', Column('reportedFrom', String(length=255), table=<routinereporttransmission>, primary_key=True, nullable=False))), ('remove_index', Index('identification', Column('seq', BIGINT(), table=<seq_element>, primary_key=True, nullable=False))), ('add_index', Index('ix_seq_element_seq', Column('seq', BigInteger(), table=<seq_element>, primary_key=True, nullable=False))), ('remove_index', Index('StationStationId', Column('stationId', VARCHAR(length=255), table=<station>, primary_key=True, nullable=False))), ('add_index', Index('ix_station_stationId', Column('stationId', String(length=255), table=<station>, primary_key=True, nullable=False))), [('modify_nullable', None, 'stationelement', 'recordedFrom', {'existing_type': VARCHAR(length=255), 'existing_server_default': False, 'existing_comment': None}, True, False)], [('modify_nullable', None, 'stationelement', 'describedBy', {'existing_type': BIGINT(), 'existing_server_default': False, 'existing_comment': None}, True, False)], [('modify_nullable', None, 'stationelement', 'recordedWith', {'existing_type': VARCHAR(length=255), 'existing_server_default': False, 'existing_comment': None}, True, False)], [('modify_nullable', None, 'stationelement', 'beginDate', {'existing_type': VARCHAR(length=50), 'existing_server_default': False, 'existing_comment': None}, True, False)], ('remove_index', Index('obsElementobservationInitial', Column('describedBy', BIGINT(), table=<stationelement>))), ('remove_index', Index('stationobservationInitial', Column('recordedFrom', VARCHAR(length=255), table=<stationelement>))), ('add_index', Index('ix_stationelement_describedBy', Column('describedBy', BigInteger(), table=<stationelement>, primary_key=True, nullable=False))), ('add_index', Index('ix_stationelement_recordedFrom', Column('recordedFrom', String(length=255), table=<stationelement>, primary_key=True, nullable=False))), ('add_index', Index('ix_stationelement_recordedWith', Column('recordedWith', String(length=255), table=<stationelement>, primary_key=True, nullable=False))), ('add_index', Index('ix_stationelement_scheduledFor', Column('scheduledFor', String(length=255), table=<stationelement>))), [('modify_nullable', None, 'stationidalias', 'idAlias', {'existing_type': VARCHAR(length=255), 'existing_server_default': False, 'existing_comment': None}, True, False)], ('remove_index', Index('stationstationidAlias', Column('belongsTo', VARCHAR(length=255), table=<stationidalias>))), ('add_index', Index('ix_stationidalias_belongsTo', Column('belongsTo', String(length=255), table=<stationidalias>))), [('modify_nullable', None, 'stationlocationhistory', 'belongsTo', {'existing_type': VARCHAR(length=255), 'existing_server_default': False, 'existing_comment': None}, True, False)], [('modify_nullable', None, 'stationlocationhistory', 'openingDatetime', {'existing_type': VARCHAR(length=50), 'existing_server_default': False, 'existing_comment': None}, True, False)], [('modify_nullable', None, 'stationqualifier', 'qualifier', {'existing_type': VARCHAR(length=255), 'existing_server_default': False, 'existing_comment': None}, True, False)], [('modify_nullable', None, 'stationqualifier', 'qualifierBeginDate', {'existing_type': VARCHAR(length=50), 'existing_server_default': False, 'existing_comment': None}, True, False)], [('modify_nullable', None, 'stationqualifier', 'qualifierEndDate', {'existing_type': VARCHAR(length=50), 'existing_server_default': False, 'existing_comment': None}, True, False)], [('modify_nullable', None, 'stationqualifier', 'belongsTo', {'existing_type': VARCHAR(length=255), 'existing_server_default': False, 'existing_comment': None}, True, False)], ('remove_index', Index('stationQualifierIdentification', Column('belongsTo', VARCHAR(length=255), table=<stationqualifier>))), ('add_index', Index('ix_stationqualifier_belongsTo', Column('belongsTo', String(length=255), table=<stationqualifier>, primary_key=True, nullable=False))), ('add_index', Index('ix_tblproducts_productId', Column('productId', String(length=10), table=<tblproducts>, primary_key=True, nullable=False))), ('add_index', Index('ix_tm_307080_synop_code', Column('synop_code', String(length=255), table=<tm_307080>))), ('add_index', Index('ix_tm_307081_synop_code', Column('synop_code', String(length=255), table=<tm_307081>))), ('add_index', Index('ix_tm_307082_synop_code', Column('synop_code', String(length=255), table=<tm_307082>))), ('add_index', Index('ix_tm_307083_synop_code', Column('synop_code', String(length=255), table=<tm_307083>))), ('add_index', Index('ix_tm_307084_synop_code', Column('synop_code', String(length=255), table=<tm_307084>))), ('add_index', Index('ix_tm_307086_synop_code', Column('synop_code', String(length=255), table=<tm_307086>))), ('remove_index', Index('synop_code', Column('Element_Name', VARCHAR(length=255), table=<tm_307089>))), ('remove_index', Index('synop_code1', Column('synop_code', VARCHAR(length=50), table=<tm_307089>))), ('add_index', Index('ix_tm_307089_Element_Name', Column('Element_Name', String(length=255), table=<tm_307089>))), ('add_index', Index('ix_tm_307089_synop_code', Column('synop_code', String(length=50), table=<tm_307089>)))]

test_db_schema_diff

[('add_table', Table('aws_lsi', MetaData(), Column('Cols', Integer(), table=<aws_lsi>, primary_key=True, nullable=False), Column('Element_abbreviation', String(length=20), table=<aws_lsi>), Column('Element_Name', String(length=20), table=<aws_lsi>), Column('Element_Details', String(length=25), table=<aws_lsi>), Column('Climsoft_Element', String(length=6), table=<aws_lsi>), Column('Bufr_Element', String(length=6), table=<aws_lsi>), Column('unit', String(length=15), table=<aws_lsi>), Column('lower_limit', String(length=10), table=<aws_lsi>), Column('upper_limit', String(length=10), table=<aws_lsi>), Column('obsv', String(length=25), table=<aws_lsi>), schema=None)), ('add_table', Table('aws_lsi1', MetaData(), Column('Cols', Integer(), table=<aws_lsi1>, primary_key=True, nullable=False), Column('Element_abbreviation', String(length=20), table=<aws_lsi1>), Column('Element_Name', String(length=20), table=<aws_lsi1>), Column('Element_Details', String(length=25), table=<aws_lsi1>), Column('Climsoft_Element', String(length=6), table=<aws_lsi1>), Column('Bufr_Element', String(length=6), table=<aws_lsi1>), Column('unit', String(length=15), table=<aws_lsi1>), Column('lower_limit', String(length=10), table=<aws_lsi1>), Column('upper_limit', String(length=10), table=<aws_lsi1>), Column('obsv', String(length=25), table=<aws_lsi1>), schema=None)), ('add_table', Table('aws_rema1', MetaData(), Column('Cols', Integer(), table=<aws_rema1>, primary_key=True, nullable=False), Column('Element_abbreviation', String(length=20), table=<aws_rema1>), Column('Element_Name', String(length=20), table=<aws_rema1>), Column('Element_Details', String(length=25), table=<aws_rema1>), Column('Climsoft_Element', String(length=6), table=<aws_rema1>), Column('Bufr_Element', String(length=6), table=<aws_rema1>), Column('unit', String(length=15), table=<aws_rema1>), Column('lower_limit', String(length=10), table=<aws_rema1>), Column('upper_limit', String(length=10), table=<aws_rema1>), Column('obsv', String(length=25), table=<aws_rema1>), schema=None)), ('add_table', Table('aws_rwanda_rain', MetaData(), Column('Cols', Integer(), table=<aws_rwanda_rain>, primary_key=True, nullable=False), Column('Element_abbreviation', String(length=20), table=<aws_rwanda_rain>), Column('Element_Name', String(length=20), table=<aws_rwanda_rain>), Column('Element_Details', String(length=25), table=<aws_rwanda_rain>), Column('Climsoft_Element', String(length=6), table=<aws_rwanda_rain>), Column('Bufr_Element', String(length=6), table=<aws_rwanda_rain>), Column('unit', String(length=15), table=<aws_rwanda_rain>), Column('lower_limit', String(length=10), table=<aws_rwanda_rain>), Column('upper_limit', String(length=10), table=<aws_rwanda_rain>), Column('obsv', String(length=25), table=<aws_rwanda_rain>), schema=None)), ('add_table', Table('aws_test', MetaData(), Column('Cols', Integer(), table=<aws_test>, nullable=False), Column('Element_abbreviation', String(length=20), table=<aws_test>), Column('Element_Name', String(length=20), table=<aws_test>), Column('Element_Details', String(length=25), table=<aws_test>), Column('Climsoft_Element', String(length=6), table=<aws_test>), Column('Bufr_Element', String(length=6), table=<aws_test>), Column('unit', String(length=15), table=<aws_test>), Column('lower_limit', String(length=10), table=<aws_test>), Column('upper_limit', String(length=10), table=<aws_test>), Column('obsv', String(length=25), table=<aws_test>), schema=None)), ('add_table', Table('gaps', MetaData(), Column('Missing_STNID', String(length=255), table=<gaps>, primary_key=True, nullable=False), Column('Missing_ELEM', BigInteger(), table=<gaps>, primary_key=True, nullable=False), Column('Missing_Date', Date(), table=<gaps>, primary_key=True, nullable=False), schema=None)), ('add_table', Table('missing_data', MetaData(), Column('STN_ID', String(length=255), table=<missing_data>, primary_key=True, nullable=False), Column('OBS_DATE', Date(), table=<missing_data>, primary_key=True, nullable=False), Column('ELEM', BigInteger(), table=<missing_data>, primary_key=True, nullable=False), schema=None)), ('add_table', Table('qcabslimits', MetaData(), Column('StationId', String(length=15), table=<qcabslimits>, primary_key=True, nullable=False), Column('ElementId', BigInteger(), table=<qcabslimits>, primary_key=True, nullable=False), Column('Datetime', DateTime(), table=<qcabslimits>, primary_key=True, nullable=False), Column('YYYY', Integer(), table=<qcabslimits>), Column('mm', TINYINT(), table=<qcabslimits>), Column('dd', TINYINT(), table=<qcabslimits>), Column('hh', TINYINT(), table=<qcabslimits>), Column('obsValue', String(length=10), table=<qcabslimits>), Column('limitValue', String(length=10), table=<qcabslimits>), Column('qcStatus', Integer(), table=<qcabslimits>), Column('acquisitionType', Integer(), table=<qcabslimits>), Column('obsLevel', String(length=255), table=<qcabslimits>), Column('capturedBy', String(length=255), table=<qcabslimits>), Column('dataForm', String(length=255), table=<qcabslimits>), schema=None)), ('add_index', Index('obsInitialIdentification', Column('StationId', String(length=15), table=<qcabslimits>, primary_key=True, nullable=False), Column('ElementId', BigInteger(), table=<qcabslimits>, primary_key=True, nullable=False), Column('Datetime', DateTime(), table=<qcabslimits>, primary_key=True, nullable=False), unique=True)), ('add_table', Table('routinereportdefinition', MetaData(), Column('reportClass', String(length=255), table=<routinereportdefinition>, primary_key=True, nullable=False), Column('reportSchedule', String(length=255), table=<routinereportdefinition>), Column('reportCode', String(length=255), table=<routinereportdefinition>), Column('reportDescription', String(length=255), table=<routinereportdefinition>), schema=None)), ('add_index', Index('ix_routinereportdefinition_reportCode', Column('reportCode', String(length=255), table=<routinereportdefinition>))), ('add_table', Table('ss', MetaData(), Column('Cols', Integer(), table=<ss>, nullable=False), Column('Element_abbreviation', String(length=20), table=<ss>), Column('Element_Name', String(length=20), table=<ss>), Column('Element_Details', String(length=25), table=<ss>), Column('Climsoft_Element', String(length=6), table=<ss>), Column('Bufr_Element', String(length=6), table=<ss>), Column('unit', String(length=15), table=<ss>), Column('lower_limit', String(length=10), table=<ss>), Column('upper_limit', String(length=10), table=<ss>), Column('obsv', String(length=25), table=<ss>), schema=None)), ('add_table', Table('testing_aws', MetaData(), Column('Cols', Integer(), table=<testing_aws>, nullable=False), Column('Element_abbreviation', String(length=20), table=<testing_aws>), Column('Element_Name', String(length=20), table=<testing_aws>), Column('Element_Details', String(length=25), table=<testing_aws>), Column('Climsoft_Element', String(length=6), table=<testing_aws>), Column('Bufr_Element', String(length=6), table=<testing_aws>), Column('unit', String(length=15), table=<testing_aws>), Column('lower_limit', String(length=10), table=<testing_aws>), Column('upper_limit', String(length=10), table=<testing_aws>), Column('obsv', String(length=25), table=<testing_aws>), schema=None)), ('add_table', Table('testing_aws1', MetaData(), Column('Cols', Integer(), table=<testing_aws1>, nullable=False), Column('Element_abbreviation', String(length=20), table=<testing_aws1>), Column('Element_Name', String(length=20), table=<testing_aws1>), Column('Element_Details', String(length=25), table=<testing_aws1>), Column('Climsoft_Element', String(length=6), table=<testing_aws1>), Column('Bufr_Element', String(length=6), table=<testing_aws1>), Column('unit', String(length=15), table=<testing_aws1>), Column('lower_limit', String(length=10), table=<testing_aws1>), Column('upper_limit', String(length=10), table=<testing_aws1>), Column('obsv', String(length=25), table=<testing_aws1>), schema=None)), ('add_table', Table('routinereporttransmission', MetaData(), Column('reportClass', String(length=255), ForeignKey('routinereportdefinition.reportClass'), table=<routinereporttransmission>, primary_key=True, nullable=False), Column('reportDatetime', DateTime(), table=<routinereporttransmission>, primary_key=True, nullable=False), Column('receivedDatetime', DateTime(), table=<routinereporttransmission>), Column('reportedFrom', String(length=255), ForeignKey('station.stationId'), table=<routinereporttransmission>, primary_key=True, nullable=False), schema=None)), ('add_index', Index('ix_routinereporttransmission_reportedFrom', Column('reportedFrom', String(length=255), table=<routinereporttransmission>, primary_key=True, nullable=False))), ('add_index', Index('report', Column('reportClass', String(length=255), table=<routinereporttransmission>, primary_key=True, nullable=False), Column('reportDatetime', DateTime(), table=<routinereporttransmission>, primary_key=True, nullable=False), Column('reportedFrom', String(length=255), table=<routinereporttransmission>, primary_key=True, nullable=False), unique=True)), ('remove_index', Index('Identification', Column('Element_Name', VARCHAR(length=50), table=<aws_malawi1>))), ('add_index', Index('ix_aws_malawi1_Element_Name', Column('Element_Name', String(length=50), table=<aws_malawi1>))), ('remove_index', Index('identification', Column('Element_Name', VARCHAR(length=50), table=<aws_rwanda1>))), ('add_index', Index('ix_aws_rwanda1_Element_Name', Column('Element_Name', String(length=50), table=<aws_rwanda1>))), [('modify_nullable', None, 'faultresolution', 'resolvedDatetime', {'existing_type': VARCHAR(length=50), 'existing_server_default': False, 'existing_comment': None}, True, False)], [('modify_nullable', None, 'faultresolution', 'associatedWith', {'existing_type': BIGINT(), 'existing_server_default': False, 'existing_comment': None}, True, False)], ('add_index', Index('ix_faultresolution_associatedWith', Column('associatedWith', BigInteger(), table=<faultresolution>, primary_key=True, nullable=False))), [('modify_nullable', None, 'featuregeographicalposition', 'observedOn', {'existing_type': VARCHAR(length=50), 'existing_server_default': False, 'existing_comment': None}, True, False)], ('remove_index', Index('code', Column('instrumentId', VARCHAR(length=255), table=<instrument>, primary_key=True, nullable=False))), ('add_index', Index('ix_instrument_installedAt', Column('installedAt', String(length=255), table=<instrument>))), ('add_index', Index('ix_instrument_instrumentId', Column('instrumentId', String(length=255), table=<instrument>, primary_key=True, nullable=False))), ('remove_index', Index('report_id', Column('reportId', BIGINT(), table=<instrumentfaultreport>, primary_key=True, nullable=False))), ('add_index', Index('ix_instrumentfaultreport_reportId', Column('reportId', BigInteger(), table=<instrumentfaultreport>, primary_key=True, nullable=False))), ('add_index', Index('ix_instrumentfaultreport_reportedFrom', Column('reportedFrom', String(length=255), table=<instrumentfaultreport>))), [('modify_nullable', None, 'instrumentinspection', 'performedOn', {'existing_type': VARCHAR(length=255), 'existing_server_default': False, 'existing_comment': None}, True, False)], [('modify_nullable', None, 'instrumentinspection', 'inspectionDatetime', {'existing_type': VARCHAR(length=50), 'existing_server_default': False, 'existing_comment': None}, True, False)], ('add_index', Index('ix_instrumentinspection_performedAt', Column('performedAt', String(length=255), table=<instrumentinspection>))), ('remove_index', Index('elementCode', Column('elementId', BIGINT(), table=<obselement>, primary_key=True, nullable=False, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x7f72156a4dc0>, for_update=False)))), ('add_index', Index('ix_obselement_elementId', Column('elementId', BigInteger(), table=<obselement>, primary_key=True, nullable=False, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x7f7216396970>, for_update=False)))), [('modify_nullable', None, 'observationfinal', 'describedBy', {'existing_type': BIGINT(), 'existing_server_default': False, 'existing_comment': None}, True, False)], [('modify_nullable', None, 'observationfinal', 'obsDatetime', {'existing_type': DATETIME(), 'existing_server_default': False, 'existing_comment': None}, True, False)], [('modify_nullable', None, 'observationfinal', 'obsLevel', {'existing_type': VARCHAR(length=255), 'existing_server_default': DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x7f721569ef10>, for_update=False), 'existing_comment': None}, True, False)], ('remove_index', Index('obsElementObservationInitial', Column('describedBy', BIGINT(), table=<observationfinal>))), ('remove_index', Index('stationObservationInitial', Column('recordedFrom', VARCHAR(length=255), table=<observationfinal>, nullable=False))), ('add_index', Index('ix_observationfinal_describedBy', Column('describedBy', BigInteger(), table=<observationfinal>, primary_key=True, nullable=False))), ('add_index', Index('ix_observationfinal_recordedFrom', Column('recordedFrom', String(length=255), table=<observationfinal>, primary_key=True, nullable=False))), [('modify_nullable', None, 'observationinitial', 'describedBy', {'existing_type': BIGINT(), 'existing_server_default': False, 'existing_comment': None}, True, False)], [('modify_nullable', None, 'observationinitial', 'obsDatetime', {'existing_type': DATETIME(), 'existing_server_default': False, 'existing_comment': None}, True, False)], [('modify_nullable', None, 'observationinitial', 'obsLevel', {'existing_type': VARCHAR(length=255), 'existing_server_default': False, 'existing_comment': None}, True, False)], [('modify_nullable', None, 'observationinitial', 'qcStatus', {'existing_type': INTEGER(), 'existing_server_default': DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x7f72154ec130>, for_update=False), 'existing_comment': None}, True, False)], [('modify_nullable', None, 'observationinitial', 'acquisitionType', {'existing_type': INTEGER(), 'existing_server_default': DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x7f72154ec250>, for_update=False), 'existing_comment': None}, True, False)], ('remove_index', Index('obsElementObservationInitial', Column('describedBy', BIGINT(), table=<observationinitial>))), ('remove_index', Index('stationObservationInitial', Column('recordedFrom', VARCHAR(length=255), table=<observationinitial>, nullable=False))), ('add_index', Index('ix_observationinitial_describedBy', Column('describedBy', BigInteger(), table=<observationinitial>, primary_key=True, nullable=False))), ('add_index', Index('ix_observationinitial_recordedFrom', Column('recordedFrom', String(length=255), table=<observationinitial>, primary_key=True, nullable=False))), [('modify_nullable', None, 'observationschedule', 'classifiedInto', {'existing_type': VARCHAR(length=255), 'existing_server_default': False, 'existing_comment': None}, True, False)], [('modify_nullable', None, 'observationschedule', 'startTime', {'existing_type': VARCHAR(length=50), 'existing_server_default': False, 'existing_comment': None}, True, False)], [('modify_nullable', None, 'observationschedule', 'endTime', {'existing_type': VARCHAR(length=50), 'existing_server_default': False, 'existing_comment': None}, True, False)], ('remove_index', Index('scheduleClassIdeification', Column('scheduleClass', VARCHAR(length=255), table=<obsscheduleclass>, primary_key=True, nullable=False, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x7f72154dfeb0>, for_update=False)))), ('add_index', Index('ix_obsscheduleclass_refersTo', Column('refersTo', String(length=255), table=<obsscheduleclass>))), ('add_index', Index('ix_obsscheduleclass_scheduleClass', Column('scheduleClass', String(length=255), table=<obsscheduleclass>, primary_key=True, nullable=False, server_default=DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x7f7216180bb0>, for_update=False)))), [('modify_nullable', None, 'paperarchive', 'belongsTo', {'existing_type': VARCHAR(length=255), 'existing_server_default': False, 'existing_comment': None}, True, False)], [('modify_nullable', None, 'paperarchive', 'formDatetime', {'existing_type': DATETIME(), 'existing_server_default': False, 'existing_comment': None}, True, False)], [('modify_nullable', None, 'paperarchive', 'classifiedInto', {'existing_type': VARCHAR(length=50), 'existing_server_default': False, 'existing_comment': None}, True, False)], ('add_index', Index('ix_paperarchive_classifiedInto', Column('classifiedInto', String(length=50), table=<paperarchive>, primary_key=True, nullable=False))), ('remove_index', Index('paperarchivedef', Column('formId', VARCHAR(length=50), table=<paperarchivedefinition>, primary_key=True, nullable=False))), ('add_index', Index('ix_paperarchivedefinition_formId', Column('formId', String(length=50), table=<paperarchivedefinition>, primary_key=True, nullable=False))), [('modify_nullable', None, 'physicalfeature', 'beginDate', {'existing_type': VARCHAR(length=50), 'existing_server_default': False, 'existing_comment': None}, True, False)], [('modify_nullable', None, 'physicalfeature', 'description', {'existing_type': VARCHAR(length=255), 'existing_server_default': False, 'existing_comment': None}, True, False)], [('modify_nullable', None, 'physicalfeature', 'classifiedInto', {'existing_type': VARCHAR(length=255), 'existing_server_default': False, 'existing_comment': None}, True, False)], ('remove_index', Index('physicalFeatureidentification_idx', Column('classifiedInto', VARCHAR(length=255), table=<physicalfeature>))), ('remove_index', Index('stationfeature', Column('associatedWith', VARCHAR(length=255), table=<physicalfeature>, nullable=False))), ('add_index', Index('ix_physicalfeature_associatedWith', Column('associatedWith', String(length=255), table=<physicalfeature>, primary_key=True, nullable=False))), ('add_index', Index('ix_physicalfeature_classifiedInto', Column('classifiedInto', String(length=255), table=<physicalfeature>, primary_key=True, nullable=False))), ('remove_index', Index('stationFeatureClass', Column('featureClass', VARCHAR(length=255), table=<physicalfeatureclass>, primary_key=True, nullable=False))), ('add_index', Index('ix_physicalfeatureclass_featureClass', Column('featureClass', String(length=255), table=<physicalfeatureclass>, primary_key=True, nullable=False))), ('add_index', Index('ix_physicalfeatureclass_refersTo', Column('refersTo', String(length=255), table=<physicalfeatureclass>))), ('remove_index', Index('identification', Column('seq', BIGINT(), table=<seq_element>, primary_key=True, nullable=False))), ('add_index', Index('ix_seq_element_seq', Column('seq', BigInteger(), table=<seq_element>, primary_key=True, nullable=False))), ('remove_index', Index('StationStationId', Column('stationId', VARCHAR(length=255), table=<station>, primary_key=True, nullable=False))), ('add_index', Index('ix_station_stationId', Column('stationId', String(length=255), table=<station>, primary_key=True, nullable=False))), [('modify_nullable', None, 'stationelement', 'recordedFrom', {'existing_type': VARCHAR(length=255), 'existing_server_default': False, 'existing_comment': None}, True, False)], [('modify_nullable', None, 'stationelement', 'describedBy', {'existing_type': BIGINT(), 'existing_server_default': False, 'existing_comment': None}, True, False)], [('modify_nullable', None, 'stationelement', 'recordedWith', {'existing_type': VARCHAR(length=255), 'existing_server_default': False, 'existing_comment': None}, True, False)], [('modify_nullable', None, 'stationelement', 'beginDate', {'existing_type': VARCHAR(length=50), 'existing_server_default': False, 'existing_comment': None}, True, False)], ('remove_index', Index('obsElementobservationInitial', Column('describedBy', BIGINT(), table=<stationelement>))), ('remove_index', Index('stationobservationInitial', Column('recordedFrom', VARCHAR(length=255), table=<stationelement>))), ('add_index', Index('ix_stationelement_describedBy', Column('describedBy', BigInteger(), table=<stationelement>, primary_key=True, nullable=False))), ('add_index', Index('ix_stationelement_recordedFrom', Column('recordedFrom', String(length=255), table=<stationelement>, primary_key=True, nullable=False))), ('add_index', Index('ix_stationelement_recordedWith', Column('recordedWith', String(length=255), table=<stationelement>, primary_key=True, nullable=False))), ('add_index', Index('ix_stationelement_scheduledFor', Column('scheduledFor', String(length=255), table=<stationelement>))), [('modify_nullable', None, 'stationidalias', 'idAlias', {'existing_type': VARCHAR(length=255), 'existing_server_default': False, 'existing_comment': None}, True, False)], ('remove_index', Index('stationstationidAlias', Column('belongsTo', VARCHAR(length=255), table=<stationidalias>))), ('add_index', Index('ix_stationidalias_belongsTo', Column('belongsTo', String(length=255), table=<stationidalias>))), [('modify_nullable', None, 'stationlocationhistory', 'belongsTo', {'existing_type': VARCHAR(length=255), 'existing_server_default': False, 'existing_comment': None}, True, False)], [('modify_nullable', None, 'stationlocationhistory', 'openingDatetime', {'existing_type': VARCHAR(length=50), 'existing_server_default': False, 'existing_comment': None}, True, False)], [('modify_nullable', None, 'stationqualifier', 'qualifier', {'existing_type': VARCHAR(length=255), 'existing_server_default': False, 'existing_comment': None}, True, False)], [('modify_nullable', None, 'stationqualifier', 'qualifierBeginDate', {'existing_type': VARCHAR(length=50), 'existing_server_default': False, 'existing_comment': None}, True, False)], [('modify_nullable', None, 'stationqualifier', 'qualifierEndDate', {'existing_type': VARCHAR(length=50), 'existing_server_default': False, 'existing_comment': None}, True, False)], [('modify_nullable', None, 'stationqualifier', 'belongsTo', {'existing_type': VARCHAR(length=255), 'existing_server_default': False, 'existing_comment': None}, True, False)], ('remove_index', Index('stationQualifierIdentification', Column('belongsTo', VARCHAR(length=255), table=<stationqualifier>))), ('add_index', Index('ix_stationqualifier_belongsTo', Column('belongsTo', String(length=255), table=<stationqualifier>, primary_key=True, nullable=False))), ('add_index', Index('ix_tblproducts_productId', Column('productId', String(length=10), table=<tblproducts>, primary_key=True, nullable=False))), ('add_index', Index('ix_tm_307080_synop_code', Column('synop_code', String(length=255), table=<tm_307080>))), ('add_index', Index('ix_tm_307081_synop_code', Column('synop_code', String(length=255), table=<tm_307081>))), ('add_index', Index('ix_tm_307082_synop_code', Column('synop_code', String(length=255), table=<tm_307082>))), ('add_index', Index('ix_tm_307083_synop_code', Column('synop_code', String(length=255), table=<tm_307083>))), ('add_index', Index('ix_tm_307084_synop_code', Column('synop_code', String(length=255), table=<tm_307084>))), ('add_index', Index('ix_tm_307086_synop_code', Column('synop_code', String(length=255), table=<tm_307086>))), ('remove_index', Index('synop_code', Column('Element_Name', VARCHAR(length=255), table=<tm_307089>))), ('remove_index', Index('synop_code1', Column('synop_code', VARCHAR(length=50), table=<tm_307089>))), ('add_index', Index('ix_tm_307089_Element_Name', Column('Element_Name', String(length=255), table=<tm_307089>))), ('add_index', Index('ix_tm_307089_synop_code', Column('synop_code', String(length=50), table=<tm_307089>))), ('remove_column', None, 'userrecords', Column('perform', INTEGER(), table=<userrecords>))]

[isedwards]

@faysal-ishtiaq - it's going to take me a while to analyse this.

One of the reasons why the two outputs differ is that they currently contain object identities (like DefaultClause(<sqlalchemy.sql.elements.TextClause object at 0x7fcd7f5a3ac8>).

I'll pformat the output and use regular expressions to remove the identity and then do a diff to see what else is different.

[faysal-ishtiaq]

@isedwards Please let me know if I can help in any way.

[isedwards]

@Shaibujnr could you help out a bit here? We're working on opencdms-dev/pyopencdms#2.

1. The first thing that we need to check is whether we can achieve is a set of models from sqlacodegen that represent the Climsoft 4.1.1 schema before we modify it in any way (the original sqlacodegen models can be seen in b3fbb85opencdms-dev/pyopencdms@b3fbb85).

   Ideally, this means that get_schema_diff() should return []. This probably requires modifying the indexes that are generated by sqlacodegen like you did with CliDE (currently documented here).

2. If we can achieve an empty list from get_schema_diff() then we'd like to modify the Climsoft DDL with the alter table commands above (see the second version of the update script here). The models created by sqlacodegen as a result of those alter table statements can be seen in 0b9c1ca.

   Once the database has been altered, we'd like to create updated models from sqlacodegen that provide us with ORM classes for the additional tables that now have primary keys.

There are a couple more steps after this, but I was hoping you could look at those first to save me a bit of time?

Could you add some comments at each step with information on what you discover as you try this? If there are still differences reported by get_schema_diff() then this may be acceptable (depending on why the differences are being reported).

[isedwards]

Models for the core of Climsoft have been added in #28 - the SQL ALTER commands and other information in the comments of this PR will be used in developer documentation (#16)