Merging updates from main

Signed-off-by: João Lucas de Sousa Almeida <[email protected]>

Author: Joao-L-S-Almeida

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,20 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you've considered (optional)**
+A clear and concise description of any alternative solutions or features you've considered.
+
+**Additional context (optional)**
+Add any other context or screenshots about the feature request here.

.github/ISSUE_TEMPLATE/issue-report.md

@@ -0,0 +1,34 @@
+---
+name: Issue report
+about: Create a report to help us improve
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+**Describe the issue**
+A clear and concise description of what the issue is.
+
+**To Reproduce (optional, but appreciated)**
+Steps to reproduce the behavior:
+1. Go to '...'
+2. Click on '....'
+3. Scroll down to '....'
+4. See error
+
+**Screenshots or log output (optional)**
+If applicable, add screenshots or log output to help explain your problem.
+<details><summary>Log Output</summary>
+<pre>
+Paste the log output here.
+</pre>
+</details>
+
+**Expected behavior (optional)**
+A clear and concise description of what you expected to happen.
+
+**Deployment information (optional)**
+Describe what you've deployed and how:
+ - TerraTorch version: [e.g. 1.5.3]
+ - Installation source: [e.g. git, pip]

.github/ISSUE_TEMPLATE/report-a-vulnerability.md

@@ -0,0 +1,17 @@
+---
+name: " Report a vulnerability "
+about: " Report a vulnerability "
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+### Summary
+_Short summary of the problem. Make the impact and severity as clear as possible. For example: An unsafe deserialization vulnerability allows any unauthenticated user to execute arbitrary code on the server._
+
+### Details
+_Give all details on the vulnerability. Pointing to the incriminated source code is very helpful for the maintainer._
+
+### Impact
+_What kind of vulnerability is it? Who is impacted?_

.github/workflows/crosshair.yaml

@@ -0,0 +1,33 @@
+# This workflow will install Python dependencies, run tests and lint with a single version of Python
+# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python
+
+name: Dynamic Code Analysis
+
+on:
+  push:
+    branches: [ "main" ]
+  pull_request:
+    branches: [ "main" ]
+
+permissions:
+  contents: read
+
+jobs:
+  build:
+
+    runs-on: ubuntu-latest
+
+    steps:
+    - uses: actions/checkout@v3
+    - name: Set up Python 3.11
+      uses: actions/setup-python@v3
+      with:
+        python-version: "3.11"
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        pip install crosshair-tool
+         pip install -r requirements/required.txt
+    - name: Test with pytest
+      run: |
+        crosshair watch ./terratorch

.github/workflows/pylint.yml

@@ -0,0 +1,28 @@
+name: Static Code Analysis (Pylint)
+
+on:
+  push:
+    branches: [ "main" ]
+  pull_request:
+    branches: [ "main" ]
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: ["3.10","3.11"]
+    steps:
+    - uses: actions/checkout@v3
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v3
+      with:
+        python-version: ${{ matrix.python-version }}
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        pip install pylint
+    - name: Analysing the code with pylint
+      run: |
+        pylint $(git ls-files '*.py')
+        pylint $(git ls-files '*.ipynb')

.gitignore

@@ -7,3 +7,5 @@ dist/*
 **/*.pt
 *.ipynb_checkpoints
 **/*pth
+.venv/*
+venv/*

CODEOWNERS

@@ -0,0 +1,5 @@
+@CarlosGomes98
+@Joao-L-S-Almeida
+@PedroConrado
+@biancazadrozny
+@romeokienzler

CODE_OF_CONDUCT.md

@@ -0,0 +1,70 @@
+## TerraTorch Community Code of Conduct 1.0
+
+
+### Our Pledge
+
+As contributors and maintainers of this project, and in the interest of fostering
+an open and welcoming community, we pledge to respect all people who contribute
+through reporting issues, posting feature requests, updating documentation,
+submitting pull requests or patches, and other activities.
+
+We are committed to making participation in this project a harassment-free experience for
+everyone, regardless of level of experience, gender, gender identity and expression,
+sexual orientation, disability, personal appearance, body size, race, ethnicity, age,
+religion, or nationality.
+
+We pledge to act and interact in ways that contribute to an open, welcoming, diverse,
+inclusive, and healthy community.
+
+### Our Standards
+
+Examples of behavior that contributes to a positive environment for our community include:
+
+* Demonstrating empathy and kindness toward other people
+* Being respectful of differing opinions, viewpoints, and experiences
+* Giving and gracefully accepting constructive feedback
+* Accepting responsibility and apologizing to those affected by our mistakes, and learning from the experience
+* Focusing on what is best not just for us as individuals, but for the overall community
+
+Examples of unacceptable behavior include:
+
+* The use of sexualized language or imagery, and sexual attention or
+  advances of any kind
+* Trolling, insulting or derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or email
+  address, without their explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Enforcement Responsibilities
+
+Community leaders are responsible for clarifying and enforcing our standards of acceptable behavior
+and will take appropriate and fair corrective action in response to any behavior that they deem
+inappropriate, threatening, offensive, or harmful.
+
+Community leaders have the right and responsibility to remove, edit, or reject comments, commits,
+code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct,
+and will communicate reasons for moderation decisions when appropriate.
+
+## Scope
+
+This Code of Conduct applies within all community spaces including Matrix, issue trackers, wikis,
+blogs, Twitter, and any other communication channels used by our community, and also applies when
+an individual is officially representing the community in public spaces. Examples of representing
+our community include using an official e-mail address, posting via an official social media account,
+or acting as an appointed representative at an online or offline event.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be reported to the community
+leaders responsible for enforcement via Matrix channel to the CLAIMED Project Management Committee at
+#claimed-pmc:matrix.org. All complaints will be reviewed and investigated promptly and fairly.
+
+All community leaders are obligated to respect the privacy and security of the reporter of any incident.
+
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant](https://www.contributor-covenant.org/),
+version 2.0, available at https://www.contributor-covenant.org/version/2/0/code_of_conduct.html.

CONTRIBUTING.md

@@ -0,0 +1,32 @@
+<!--
+{% comment %}
+Copyright 2018-2024 IBM
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+{% endcomment %}
+-->
+
+# Contributing
+
+Welcome to TerraTorch! If you are interested in contributing to the [TerraTorch code repo](README.md)
+then checkout the [Contribution Process](contribution_process.md) and 
+the [Code of Conduct](CODE_OF_CONDUCT.md)). 
+
+The [erraTorch code repo]([https://github.com/IBM/terratorch/blob/main/README.md)) contains information on how the community
+is organized and other information that is pertinent to contributing.
+
+### Getting Started
+
+It's encouraged that you look under the [Issues]([https://github.com/IBM/terratorch/issues)) tab for contribution opportunites.
+
+Please also take the opportunity to join our [weekly community call](https://teams.microsoft.com/l/meetup-join/19%3ameeting_MWJhMThhMTMtMjc3MS00YjAyLWI3NTMtYTI0NDQ3NWY3ZGU2%40thread.v2/0?context=%7b%22Tid%22%3a%22fcf67057-50c9-4ad4-98f3-ffca64add9e9%22%2c%22Oid%22%3a%227f7ab87a-680c-4c93-acc5-fbd7ec80823a%22%7d) every Thursday 7 AM PST, 10 AM EST, **2 PM UTC**, 4 PM CET, 7:30 PM IST, 11 PM JST, 12 AM AEST

VULNERABILITIES.md

@@ -0,0 +1,4 @@
+# VULNERABILITIES reporting process
+
+Vulnerabilities can be reported on the GitHub Issue Tracker. 
+We are working on a more generic channel and update this information accordingly

contribution_process.md

@@ -0,0 +1,9 @@
+# Contribution Process
+
+If you want to contribute to this project, there are many valuable ways in doing so
+
+1. Join the weekly community calls as indicated in [CONTRIBUTING.md](CONTRIBUTING.md)
+1. Use / test TerraTorch and create an [Issue](https://github.com/IBM/terratorch/issues) if something is not working properly or if you have an idea for a feature request.
+1. Pick an [Issue](https://github.com/IBM/terratorch/issues) and start contributing
+
+Contributions are welcome as pull requests on a [fork](https://github.com/IBM/terratorch/fork) of this project. Ideally, pull requests are backed by an [Issue](https://github.com/IBM/terratorch/issues). You can also tag the [code owners](https://github.com/IBM/terratorch/blob/main/CODEOWNERS) in the issue before you start, so we can talk about the details (in case you can't join one of the community calls).

docs/models.md

@@ -61,6 +61,7 @@ By passing a list of bands being used to the constructor, we automatically filte
 
 ## Model Factory
 ### :::terratorch.models.PrithviModelFactory
+### :::terratorch.models.SMPModelFactory
 
 # Adding new model types
 Adding new model types is as simple as creating a new factory that produces models. See for instance the example below for a potential `SMPModelFactory`

pyproject.toml

@@ -43,7 +43,9 @@ dependencies = [
   "geobench>=1.0.0",
   "mlflow>=2.12.1",
   # broken due to https://github.com/Lightning-AI/pytorch-lightning/issues/19977
-  "lightning>=2, <=2.2.5"
+  "lightning>=2, <=2.2.5",
+  # see issue #64
+  "albumentations<=1.4.10"
 ]
 
 [project.optional-dependencies]

terratorch/cli_tools.py

@@ -123,7 +123,7 @@ def write_on_epoch_end(self, trainer, pl_module, predictions, batch_indices):  #
             output_dir = self.output_dir
 
         if not os.path.exists(output_dir):
-            os.mkdir(output_dir)
+            os.makedirs(output_dir, exist_ok=True)
 
         for pred_batch, filename_batch in predictions:
             for prediction, file_name in zip(torch.unbind(pred_batch, dim=0), filename_batch, strict=False):
@@ -467,4 +467,4 @@ def inference(self, file_path: Path) -> torch.Tensor:
             prediction, file_name = self.inference_on_dir(
                 tmpdir,
             )
-            return prediction.squeeze(0)
+            return prediction.squeeze(0)

terratorch/datasets/generic_pixel_wise_dataset.py

@@ -20,7 +20,7 @@
 from torch import Tensor
 from torchgeo.datasets import NonGeoDataset
 
-from terratorch.datasets.utils import HLSBands, filter_valid_files, to_tensor
+from terratorch.datasets.utils import HLSBands, default_transform, filter_valid_files
 
 
 class GenericPixelWiseDataset(NonGeoDataset, ABC):
@@ -136,7 +136,7 @@ def __init__(
             self.filter_indices = None
 
         # If no transform is given, apply only to transform to torch tensor
-        self.transform = transform if transform else lambda **batch: to_tensor(batch)
+        self.transform = transform if transform else default_transform
         # self.transform = transform if transform else ToTensorV2()
 
     def __len__(self) -> int:
@@ -155,14 +155,15 @@ def __getitem__(self, index: int) -> dict[str, Any]:
             "image": image.astype(np.float32) * self.constant_scale,
             "mask": self._load_file(self.segmentation_mask_files[index], nan_replace=self.no_label_replace).to_numpy()[
                 0
-            ],
-            "filename": self.image_files[index],
+            ]
         }
 
         if self.reduce_zero_label:
             output["mask"] -= 1
         if self.transform:
             output = self.transform(**output)
+        output["filename"] = self.image_files[index]
+
         return output
 
     def _load_file(self, path, nan_replace: int | float | None = None) -> xr.DataArray:
@@ -185,10 +186,6 @@ def _generate_bands_intervals(self, bands_intervals: list[int | str | HLSBands |
                 bands.extend(expanded_element)
             else:
                 bands.append(element)
-        # check the expansion didnt result in duplicate elements
-        if len(set(bands)) != len(bands):
-            msg = "Duplicate indices detected. Indices must be unique."
-            raise Exception(msg)
         return bands
 
 

terratorch/datasets/generic_scalar_label_dataset.py

@@ -26,7 +26,7 @@
 from torchgeo.datasets.utils import rasterio_loader
 from torchvision.datasets import ImageFolder
 
-from terratorch.datasets.utils import HLSBands, filter_valid_files, to_tensor
+from terratorch.datasets.utils import HLSBands, default_transform, filter_valid_files
 
 
 class GenericScalarLabelDataset(NonGeoDataset, ImageFolder, ABC):
@@ -128,7 +128,7 @@ def is_valid_file(x):
         else:
             self.filter_indices = None
         # If no transform is given, apply only to transform to torch tensor
-        self.transforms = transform if transform else lambda **batch: to_tensor(batch)
+        self.transforms = transform if transform else default_transform
         # self.transform = transform if transform else ToTensorV2()
 
     def __len__(self) -> int:
@@ -143,13 +143,12 @@ def __getitem__(self, index: int) -> dict[str, Any]:
 
         output = {
             "image": image.astype(np.float32) * self.constant_scale,
-            "label": label,
-            "filename": self.samples[index][
-                0
-            ],  # samples is an attribute of ImageFolder. Contains a tuple of (Path, Target)
+            "label": label,  # samples is an attribute of ImageFolder. Contains a tuple of (Path, Target)
         }
         if self.transforms:
             output = self.transforms(**output)
+        output["filename"] = self.image_files[index]
+
         return output
 
     def _load_file(self, path) -> xr.DataArray: