|
1 | | -A Simple Python Project Skeleton |
2 | | -================================ |
| 1 | +aboutcode.federated |
| 2 | +=================== |
3 | 3 |
|
4 | | -This repo attempts to standardize the structure of the Python-based project's |
5 | | -repositories using modern Python packaging and configuration techniques. |
6 | | -Using this `blog post`_ as inspiration, this repository serves as the base for |
7 | | -all new Python projects and is mergeable in existing repositories as well. |
| 4 | +This is a library of utilities to compute ids and file paths for AboutCode |
| 5 | +federated data based on Package URL |
8 | 6 |
|
9 | | -.. _blog post: https://blog.jaraco.com/a-project-skeleton-for-python-projects/ |
10 | 7 |
|
| 8 | +Federated data utilities goal is to handle content-defined and hash-addressable |
| 9 | +Package data keyed by PURL stored in many Git repositories. This approach to |
| 10 | +federate decentralized data is called FederatedCode. |
11 | 11 |
|
12 | | -Usage |
13 | | -===== |
14 | 12 |
|
15 | | -A brand new project |
16 | | -------------------- |
| 13 | +Overview |
| 14 | +======== |
17 | 15 |
|
18 | | -.. code-block:: bash |
| 16 | +The main design elements for these utilities are: |
19 | 17 |
|
20 | | - git init my-new-repo |
21 | | - cd my-new-repo |
22 | | - git pull git@github.com:nexB/skeleton |
| 18 | +1. **Data Federation**: A Data Federation is a database, representing a consistent, |
| 19 | +non-overlapping set of data kind clusters (like scans, vulnerabilities or SBOMs) |
| 20 | +across many package ecosystems, aka. PURL types. |
| 21 | +A Federation is similar to a traditional database. |
23 | 22 |
|
24 | | - # Create the new repo on GitHub, then update your remote |
25 | | - git remote set-url origin git@github.com:nexB/your-new-repo.git |
| 23 | +2. **Data Cluster**: A Data Federation contains Data Clusters, where a Data Cluster |
| 24 | +purpose is to store the data of a single kind (like scans) across multiple PURL |
| 25 | +types. The cluster name is the data kind name and is used as the prefix for |
| 26 | +repository names. A Data Cluster is akin to a table in a traditional database. |
26 | 27 |
|
27 | | -From here, you can make the appropriate changes to the files for your specific project. |
| 28 | +3. **Data Repository**: A DataCluster contains of one or more Git Data Repository, |
| 29 | +each storing datafiles of the cluster data kind and a one PURL type, spreading |
| 30 | +the datafiles in multiple Data Directories. The name is data-kind +PURL- |
| 31 | +type+hashid. A Repository is similar to a shard or tablespace in a traditionale |
| 32 | +database. |
28 | 33 |
|
29 | | -Update an existing project |
30 | | ---------------------------- |
| 34 | +4. **Data Directory**: In a Repository, a Data Directory contains the datafiles for |
| 35 | +PURLs. The directory name PURL-type+hashid |
31 | 36 |
|
32 | | -.. code-block:: bash |
| 37 | +5. **Data File**: This is a Data File of the DataCluster's Data Kind that is |
| 38 | +stored in subdirectories structured after the PURL components:: |
33 | 39 |
|
34 | | - cd my-existing-project |
35 | | - git remote add skeleton git@github.com:nexB/skeleton |
36 | | - git fetch skeleton |
37 | | - git merge skeleton/main --allow-unrelated-histories |
| 40 | + namespace/name/version/qualifiers/subpath: |
38 | 41 |
|
39 | | -This is also the workflow to use when updating the skeleton files in any given repository. |
| 42 | +- Either at the level of a PURL name: namespace/name, |
| 43 | +- Or at the PURL version level namespace/name/version, |
| 44 | +- Or at the PURL qualifiers+PURL subpath level. |
40 | 45 |
|
41 | | -More usage instructions can be found in ``docs/skeleton-usage.rst``. |
| 46 | +A Data File can be for instance a JSON scan results file, or a list of PURLs in |
| 47 | +YAML. |
42 | 48 |
|
| 49 | +For example, a list of PURLs as a Data Kind would stored at the name |
| 50 | +subdirectory level:: |
43 | 51 |
|
44 | | -Release Notes |
45 | | -============= |
| 52 | + gem-0107/gem/random_password_generator/purls.yml |
46 | 53 |
|
47 | | -- 2025-03-31: |
| 54 | +Or a ScanCode scan as a Data Kind at the version subdirectory level:: |
48 | 55 |
|
49 | | - - Use ruff as the main code formatting tool, add ruff rules to pyproject.toml |
| 56 | + gem-0107/npm/file/3.24.3/scancode.yml |
50 | 57 |
|
51 | | -- 2025-03-29: |
52 | 58 |
|
53 | | - - Add support for beta macOS-15 |
54 | | - - Add support for beta windows-2025 |
55 | 59 |
|
56 | | -- 2025-02-14: |
| 60 | +License |
| 61 | +------- |
57 | 62 |
|
58 | | - - Drop support for Python 3.8, add support in CI for Python 3.13, use Python 3.12 as default |
59 | | - version. |
| 63 | +Copyright (c) AboutCode and others. All rights reserved. |
60 | 64 |
|
61 | | -- 2025-01-17: |
| 65 | +SPDX-License-Identifier: Apache-2.0 |
62 | 66 |
|
63 | | - - Drop support for macOS-12, add support for macOS-14 |
64 | | - - Add support in CI for ubuntu-24.04 |
65 | | - - Add support in CI for Python 3.12 |
| 67 | +See https://github.com/aboutcode-org/vulnerablecode for support or download. |
66 | 68 |
|
67 | | -- 2024-08-20: |
68 | | - |
69 | | - - Update references of ownership from nexB to aboutcode-org |
70 | | - |
71 | | -- 2024-07-01: |
72 | | - |
73 | | - - Drop support for Python 3.8 |
74 | | - - Drop support for macOS-11, add support for macOS-14 |
75 | | - |
76 | | -- 2024-02-19: |
77 | | - |
78 | | - - Replace support in CI of default ubuntu-20.04 by ubuntu-22.04 |
79 | | - |
80 | | -- 2023-10-18: |
81 | | - |
82 | | - - Add dark mode support in documentation |
83 | | - |
84 | | -- 2023-07-18: |
85 | | - |
86 | | - - Add macOS-13 job in azure-pipelines.yml |
87 | | - |
88 | | -- 2022-03-04: |
89 | | - |
90 | | - - Synchronize configure and configure.bat scripts for sanity |
91 | | - - Update CI operating system support with latest Azure OS images |
92 | | - - Streamline utility scripts in etc/scripts/ to create, fetch and manage third-party |
93 | | - dependencies. There are now fewer scripts. See etc/scripts/README.rst for details |
94 | | - |
95 | | -- 2021-09-03: |
96 | | - - ``configure`` now requires pinned dependencies via the use of ``requirements.txt`` |
97 | | - and ``requirements-dev.txt`` |
98 | | - - ``configure`` can now accept multiple options at once |
99 | | - - Add utility scripts from scancode-toolkit/etc/release/ for use in generating project files |
100 | | - - Rename virtual environment directory from ``tmp`` to ``venv`` |
101 | | - - Update README.rst with instructions for generating ``requirements.txt`` |
102 | | - and ``requirements-dev.txt``, as well as collecting dependencies as wheels and generating |
103 | | - ABOUT files for them. |
104 | | - |
105 | | -- 2021-05-11: |
106 | | - - Adopt new configure scripts from ScanCode TK that allows correct configuration of which |
107 | | - Python version is used. |
| 69 | +See https://aboutcode.org for more information about AboutCode OSS projects. |
0 commit comments