Skip to content

Waste water data submission #448

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Jan 13, 2025
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .env.schema
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,10 @@ NEXT_PUBLIC_EGO_API_URL=
NEXT_PUBLIC_EGO_CLIENT_ID=
NEXT_PUBLIC_KEYCLOAK_API_URL=

# ###### Environmental submission
NEXT_PUBLIC_ENVIRONMENTAL_SUBMISSION_API_URL=
NEXT_PUBLIC_ENVIRONMENTAL_SUBMISSION_CATEGORY_ID=
Comment on lines +32 to +33
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

environment variables to set up Submission service

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's a category id in this context?

Copy link

@joneubank joneubank Jan 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lyric data category ID. Lyric is built to manage data of multiple data-dictionaries and multiple submitting projects, so when submitting we need to identify both which data category (data-dictionary) we are submitting for, and what our project ID is. Not sure if we have a separate env var for the project ID or not...


# ######## Muse
# for dev work, remember to add "http://localhost:3000" to Muse's CORS allowed domains (i.e. )
NEXT_PUBLIC_MUSE_API_URL=
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
import { css, useTheme } from '@emotion/react';
import Router from 'next/router';
import { ReactElement, useEffect, useReducer, useState } from 'react';
import urlJoin from 'url-join';

import { ButtonElement as Button } from '@/components/Button';
import ErrorNotification from '@/components/ErrorNotification';
Expand Down Expand Up @@ -80,7 +81,11 @@ const NewSubmissions = (): ReactElement => {

default: {
response.submissionId
? Router.push(getInternalLink({ path: `submission/${response.submissionId}` }))
? Router.push(
getInternalLink({
path: urlJoin('submission', 'clinical', response.submissionId),
}),
)
: console.log('Unhandled response:', response);
return Promise.resolve();
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ import { css } from '@emotion/react';
import { format } from 'date-fns';
import { ReactElement } from 'react';
import { Column } from 'react-table';
import urljoin from 'url-join';

import { numberSort, uuidSort } from '@/components/GenericTable/helpers';
import StyledLink from '@/components/Link';
Expand All @@ -32,7 +33,9 @@ const columnData: Column<Record<string, unknown>>[] = [
{
accessor: 'submissionId',
Cell: ({ value }: { value: unknown }) => (
<StyledLink href={getInternalLink({ path: `/submission/${value}` })}>
<StyledLink
href={getInternalLink({ path: urljoin('submission', 'clinical', String(value)) })}
>
{value as string}
</StyledLink>
),
Expand Down
40 changes: 9 additions & 31 deletions components/pages/submission/Clinical/index.tsx
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
/*
*
* Copyright (c) 2021 The Ontario Institute for Cancer Research. All rights reserved
* Copyright (c) 2024 The Ontario Institute for Cancer Research. All rights reserved
*
* This program and the accompanying materials are made available under the terms of
* the GNU Affero General Public License v3.0. You should have received a copy of the
Expand All @@ -19,38 +19,16 @@
*
*/

import { css, useTheme } from '@emotion/react';
import { ReactElement } from 'react';

import defaultTheme from '@/components/theme';
import PageLayout from '@/components/PageLayout';

import NewSubmissions from './NewSubmissions';
import PreviousSubmissions from './PreviousSubmissions';
import PageContent from './PageContent';

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did we name this component? It is an extremely vague component name.


const EnvironmentalDataSubmissionPage = (): ReactElement => {
const theme: typeof defaultTheme = useTheme();
const ClinicalDataSubmissionPage = (): ReactElement => (
<PageLayout subtitle="Submission Dashboard">
<PageContent />
</PageLayout>
);

return (
<>
<h1 className="view-title">Clinical Case Submissions</h1>

<section
css={css`
display: flex;
padding: 40px 0 calc(${theme.dimensions.footer.height}px + 30px);
position: relative;

> * {
flex-basis: 50%;
padding: 0 30px;
}
`}
>
<PreviousSubmissions />
<NewSubmissions />
</section>
</>
);
};

export default EnvironmentalDataSubmissionPage;
export default ClinicalDataSubmissionPage;
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
/*
*
* Copyright (c) 2021 The Ontario Institute for Cancer Research. All rights reserved
* Copyright (c) 2024 The Ontario Institute for Cancer Research. All rights reserved
*
* This program and the accompanying materials are made available under the terms of
* the GNU Affero General Public License v3.0. You should have received a copy of the
Expand Down Expand Up @@ -48,7 +48,7 @@ const DropZone = ({
isDragActive,
// isFileTooLarge,
} = useDropzone({
accept: '.fa,.gz,.fasta,.tsv,text/tab-separated-values',
accept: '.csv',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would it be valid/desirable to also accept tsv?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean to convert it to .csv for them? Or is this expecting that they may submit .tsv in the future and we should accept that?

disabled,
onDrop: useCallback(
(acceptedFiles: File[]) =>
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
/*
*
* Copyright (c) 2021 The Ontario Institute for Cancer Research. All rights reserved
* Copyright (c) 2024 The Ontario Institute for Cancer Research. All rights reserved
*
* This program and the accompanying materials are made available under the terms of
* the GNU Affero General Public License v3.0. You should have received a copy of the
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
/*
*
* Copyright (c) 2021 The Ontario Institute for Cancer Research. All rights reserved
* Copyright (c) 2024 The Ontario Institute for Cancer Research. All rights reserved
*
* This program and the accompanying materials are made available under the terms of
* the GNU Affero General Public License v3.0. You should have received a copy of the
Expand Down
72 changes: 35 additions & 37 deletions components/pages/submission/Environmental/NewSubmissions/index.tsx
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
/*
*
* Copyright (c) 2021 The Ontario Institute for Cancer Research. All rights reserved
* Copyright (c) 2024 The Ontario Institute for Cancer Research. All rights reserved
*
* This program and the accompanying materials are made available under the terms of
* the GNU Affero General Public License v3.0. You should have received a copy of the
Expand All @@ -22,14 +22,15 @@
import { css, useTheme } from '@emotion/react';
import Router from 'next/router';
import { ReactElement, useEffect, useReducer, useState } from 'react';
import urlJoin from 'url-join';

import { ButtonElement as Button } from '@/components/Button';
import ErrorNotification from '@/components/ErrorNotification';
import StyledLink from '@/components/Link';
import { LoaderWrapper } from '@/components/Loader';
import defaultTheme from '@/components/theme';
import useAuthContext from '@/global/hooks/useAuthContext';
import useMuseData from '@/global/hooks/useMuseData';
import useEnvironmentalData from '@/global/hooks/useEnvironmentalData';
import getInternalLink from '@/global/utils/getInternalLink';

import DropZone from './DropZone';
Expand All @@ -41,49 +42,57 @@ import { getFileExtension, validationParameters, validationReducer } from './val
const noUploadError = {} as NoUploadErrorType;

const NewSubmissions = (): ReactElement => {
const { token, userHasWriteScopes } = useAuthContext();
const { token, userHasWriteScopes, user } = useAuthContext();
const theme: typeof defaultTheme = useTheme();
const [thereAreFiles, setThereAreFiles] = useState(false);
const [uploadError, setUploadError] = useState(noUploadError);
const [validationState, validationDispatch] = useReducer(validationReducer, validationParameters);
const { oneTSV, oneOrMoreFasta, readyToUpload } = validationState;
const { oneOrMoreCsv, readyToUpload } = validationState;

const { awaitingResponse, fetchMuseData } = useMuseData('NewSubmissions');
const { awaitingResponse, submitData } = useEnvironmentalData('NewSubmissions');
// TODO: get Organization/StudyID from user context
// TEST PURPOSE ONLY. REPLACE THIS FROM USER CONTEXT
const organization = 'QC-WW';

const handleSubmit = () => {
if (thereAreFiles && token && userHasWriteScopes) {
const formData = new FormData();
formData.append('organization', organization);

// if many TSV are available, submit only the first one along with all fastas
const selectedTSV = oneTSV.slice(-1)[0];
formData.append('files', selectedTSV, selectedTSV.name);
oneOrMoreFasta.forEach((fasta) => formData.append('files', fasta, fasta.name));
oneOrMoreCsv.forEach((csvFile) => formData.append('files', csvFile, csvFile.name));

return fetchMuseData('submissions', { body: formData, method: 'POST' }).then((response) => {
return submitData({ body: formData }).then((response) => {
switch (response.status) {
case 'BAD_REQUEST': {
case 'INVALID_FILE_EXTENSION':
case 'FILE_READ_ERROR':
case 'UNRECOGNIZED_HEADER':
case 'MISSING_REQUIRED_HEADER': {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we communicate the specific error?

setUploadError({
...response,
status: 'Your submission has errors and cannot be processed.',
});
return Promise.resolve();
}

case 'INTERNAL_SERVER_ERROR': {
case 'PROCESSING': {
response.submissionId
? Router.push(
getInternalLink({
path: urlJoin('submission', 'environmental', response.submissionId.toString()),
}),
)
: console.log('Unhandled response:', response);
return Promise.resolve();
}

default: {
console.error(response);
setUploadError({
status: 'Internal server error',
message: 'Your upload request has failed. Please try again later.',
});
return Promise.resolve();
}

default: {
response.submissionId
? Router.push(getInternalLink({ path: `submission/${response.submissionId}` }))
: console.log('Unhandled response:', response);
return Promise.resolve();
}
}
});
}
Expand All @@ -93,9 +102,7 @@ const NewSubmissions = (): ReactElement => {

useEffect(() => {
setUploadError(noUploadError);
setThereAreFiles(
validationState.oneTSV.length > 0 || validationState.oneOrMoreFasta.length > 0,
);
setThereAreFiles(validationState.oneOrMoreCsv.length > 0);
}, [validationState]);

const handleClearAll = () => {
Expand Down Expand Up @@ -254,7 +261,7 @@ const NewSubmissions = (): ReactElement => {
loading={awaitingResponse}
message={
<>
Currently validating metadata and sequencing files.
Currently validating metadata files.
<br />
Do not navigate away from this browser window.
</>
Expand Down Expand Up @@ -331,21 +338,12 @@ const NewSubmissions = (): ReactElement => {
<tbody>
{thereAreFiles ? (
<>
{oneTSV.map((tsv, index) => (
// when more than one, all but the last one will get crossed out on render
<FileRow
active={index === oneTSV.length - 1}
file={tsv}
key={tsv.name}
handleRemove={handleRemoveThis(tsv)}
/>
))}
{oneOrMoreFasta.map((fasta: File) => (
{oneOrMoreCsv.map((csvFile: File) => (
<FileRow
active={true}
file={fasta}
key={fasta.name}
handleRemove={handleRemoveThis(fasta)}
file={csvFile}
key={csvFile.name}
handleRemove={handleRemoveThis(csvFile)}
/>
))}
</>
Expand Down Expand Up @@ -377,7 +375,7 @@ const NewSubmissions = (): ReactElement => {
margin-left: 10px;
`}
>
You must submit only one metadata TSV file and at least one FASTA file.
You must submit at least one CSV file.
</p>
)}
</td>
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
/*
*
* Copyright (c) 2021 The Ontario Institute for Cancer Research. All rights reserved
* Copyright (c) 2024 The Ontario Institute for Cancer Research. All rights reserved
*
* This program and the accompanying materials are made available under the terms of
* the GNU Affero General Public License v3.0. You should have received a copy of the
Expand Down Expand Up @@ -50,19 +50,18 @@ export type ReaderCallbackType = (result: string | ArrayBuffer | null) => void;

export type ValidationActionType =
| {
type: 'add fasta' | 'add tsv';
type: 'add csv';
file: File;
}
| {
type: 'remove fasta' | 'remove tsv';
type: 'remove csv';
file: string;
}
| {
type: 'clear all' | 'is ready' | 'not ready';
};

export type ValidationParametersType = {
oneTSV: File[];
oneOrMoreFasta: File[];
oneOrMoreCsv: File[];
readyToUpload: boolean;
};
Loading