Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chg: [merge] updated #688

Merged
merged 12 commits into from
Aug 22, 2024
6 changes: 4 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,7 @@ For further Information see the [license file](https://misp.github.io/misp-modul
* [Lastline Submit](https://misp.github.io/misp-modules/expansion/#lastline-submit) - Deprecation notice: this module will be deprecated by December 2021, please use vmware_nsx module. Module to submit a file or URL to Lastline.
* [Macaddress.io Lookup](https://misp.github.io/misp-modules/expansion/#macaddress.io-lookup) - MISP hover module for macaddress.io
* [Macvendors Lookup](https://misp.github.io/misp-modules/expansion/#macvendors-lookup) - Module to access Macvendors API.
* [Malshare Upload](https://misp.github.io/misp-modules/expansion/#malshare-upload) - Module to push malware samples to malshare.com .
* [Malware Bazaar Lookup](https://misp.github.io/misp-modules/expansion/#malware-bazaar-lookup) - Query Malware Bazaar to get additional information about the input hash.
* [McAfee MVISION Insights Lookup](https://misp.github.io/misp-modules/expansion/#mcafee-mvision-insights-lookup) - Lookup McAfee MVISION Insights Details
* [GeoIP Enrichment](https://misp.github.io/misp-modules/expansion/#geoip-enrichment) - A hover and expansion module to enrich an ip with geolocation and ASN information from an mmdb server instance, such as CIRCL's ip.circl.lu.
Expand Down Expand Up @@ -123,18 +124,21 @@ For further Information see the [license file](https://misp.github.io/misp-modul
* [ThreatCrowd Lookup](https://misp.github.io/misp-modules/expansion/#threatcrowd-lookup) - Module to get information from ThreatCrowd.
* [ThreadFox Lookup](https://misp.github.io/misp-modules/expansion/#threadfox-lookup) - Module to search for an IOC on ThreatFox by abuse.ch.
* [ThreatMiner Lookup](https://misp.github.io/misp-modules/expansion/#threatminer-lookup) - Module to get information from ThreatMiner.
* [Triage Submit](https://misp.github.io/misp-modules/expansion/#triage-submit) - Module to submit samples to tria.ge .
* [TruSTAR Enrich](https://misp.github.io/misp-modules/expansion/#trustar-enrich) - Module to get enrich indicators with TruSTAR.
* [URLhaus Lookup](https://misp.github.io/misp-modules/expansion/#urlhaus-lookup) - Query of the URLhaus API to get additional information about the input attribute.
* [URLScan Lookup](https://misp.github.io/misp-modules/expansion/#urlscan-lookup) - An expansion module to query urlscan.io.
* [VARIoT db Lookup](https://misp.github.io/misp-modules/expansion/#variot-db-lookup) - An expansion module to query the VARIoT db API for more information about a vulnerability.
* [VirusTotal v3 Lookup](https://misp.github.io/misp-modules/expansion/#virustotal-v3-lookup) - Enrich observables with the VirusTotal v3 API
* [VirusTotal Public API Lookup](https://misp.github.io/misp-modules/expansion/#virustotal-public-api-lookup) - Enrich observables with the VirusTotal v3 public API
* [VirusTotal Upload](https://misp.github.io/misp-modules/expansion/#virustotal-upload) - Module to push malware samples to VirusTotal v3 public API
* [VMRay Submit](https://misp.github.io/misp-modules/expansion/#vmray-submit) - Module to submit a sample to VMRay.
* [VMware NSX Defender Enrich](https://misp.github.io/misp-modules/expansion/#vmware-nsx-defender-enrich) - Module to enrich a file or URL with VMware NSX Defender.
* [VulnDB Lookup](https://misp.github.io/misp-modules/expansion/#vulndb-lookup) - Module to query VulnDB (RiskBasedSecurity.com).
* [Vulnerability Lookup](https://misp.github.io/misp-modules/expansion/#vulnerability-lookup) - An expansion module to query Vulnerability Lookup
* [Vulners Lookup](https://misp.github.io/misp-modules/expansion/#vulners-lookup) - An expansion hover module to expand information about CVE id using Vulners API.
* [Vysion Enrich](https://misp.github.io/misp-modules/expansion/#vysion-enrich) - Module to enrich the information by making use of the Vysion API.
* [Whois Lookup](https://misp.github.io/misp-modules/expansion/#whois-lookup) - Module to query a local instance of uwhois (https://github.com/rafiot/uwhoisd).
* [WhoisFreaks Lookup](https://misp.github.io/misp-modules/expansion/#whoisfreaks-lookup) - An expansion module for https://whoisfreaks.com/ that will provide an enriched analysis of the provided domain, including WHOIS and DNS information.
* [Wikidata Lookup](https://misp.github.io/misp-modules/expansion/#wikidata-lookup) - An expansion hover module to extract information from Wikidata to have additional information about particular term for analysis.
* [IBM X-Force Exchange Lookup](https://misp.github.io/misp-modules/expansion/#ibm-x-force-exchange-lookup) - An expansion module for IBM X-Force Exchange.
Expand Down Expand Up @@ -181,5 +185,3 @@ For further Information see the [license file](https://misp.github.io/misp-modul
* [Mattermost](https://misp.github.io/misp-modules/action_mod/#mattermost) - Simplistic module to send message to a Mattermost channel.
* [Slack](https://misp.github.io/misp-modules/action_mod/#slack) - Simplistic module to send messages to a Slack channel.
* [Test action](https://misp.github.io/misp-modules/action_mod/#test-action) - This module is merely a test, always returning true. Triggers on event publishing.


1 change: 1 addition & 0 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,7 @@ For more information: [Extending MISP with Python modules](https://www.misp-proj
* [Vulnerability Lookup](https://misp.github.io/misp-modules/expansion/#vulnerability-lookup) - An expansion module to query Vulnerability Lookup
* [Vulners Lookup](https://misp.github.io/misp-modules/expansion/#vulners-lookup) - An expansion hover module to expand information about CVE id using Vulners API.
* [Vysion Enrich](https://misp.github.io/misp-modules/expansion/#vysion-enrich) - Module to enrich the information by making use of the Vysion API.
* [Whois Lookup](https://misp.github.io/misp-modules/expansion/#whois-lookup) - Module to query a local instance of uwhois (https://github.com/rafiot/uwhoisd).
* [WhoisFreaks Lookup](https://misp.github.io/misp-modules/expansion/#whoisfreaks-lookup) - An expansion module for https://whoisfreaks.com/ that will provide an enriched analysis of the provided domain, including WHOIS and DNS information.
* [Wikidata Lookup](https://misp.github.io/misp-modules/expansion/#wikidata-lookup) - An expansion hover module to extract information from Wikidata to have additional information about particular term for analysis.
* [IBM X-Force Exchange Lookup](https://misp.github.io/misp-modules/expansion/#ibm-x-force-exchange-lookup) - An expansion module for IBM X-Force Exchange.
Expand Down
24 changes: 18 additions & 6 deletions docs/install.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,3 @@
IMPORTANT NOTE: we will soon be publishing `misp-modules` on PyPI.


## Install from pip

It is strongly recommended to use a virtual environment (see here for instructions https://docs.python.org/3/tutorial/venv.html).
Expand All @@ -11,6 +8,8 @@ Once the virtual environment is loaded just use the command:
pip install misp-modules
~~~~

Note: this install method might not yet be available.


## Install from cloned repository

Expand Down Expand Up @@ -83,7 +82,7 @@ Inside you will find three targets:

- `test-docs`: run a local server exposing the newly built documentation.

Note that you can either run the targets using `poetry` (default), or using the `squidfunk/mkdocs-material` by setting the environment variable `USE_DOCKER=true`.
Note: you can either run the targets using `poetry` (default), or using the Docker image `squidfunk/mkdocs-material` by setting the environment variable `USE_DOCKER=true`.


## Run MISP modules
Expand All @@ -93,7 +92,7 @@ If you installed it using pip, you just need to execute the command `misp-module

## Run MISP modules in Docker

You can find an up-to-date container image and related documentation at the following repository:m https://github.com/MISP/misp-docker
You can find an up-to-date container image and related documentation at the following repository: https://github.com/MISP/misp-docker .


## Install misp-module on an offline instance
Expand Down Expand Up @@ -137,11 +136,24 @@ Just follow those instructions but replace the package `misp-modules` with `-r r

Before doing so you need to generate the `requirements.txt` file. Due to the fact we are still supporting Python 3.8 and that Poetry still has some limitations (soon to be resolved) you need to need to replace the line `python = ">=3.8.*,<3.13"` inside `pyproject.toml` with your exact version (just run `python --version`).

Once you have done that, run the following commands to generate your very own `requirements.txt`.
The following `sed` command does everything for you.

~~~~bash
sed -i "s/^python = .*/python = \"$(python -c 'import platform; print(platform.python_version())')\"/" pyproject.toml
~~~~

Then, run the following commands to generate your very own `requirements.txt`.

~~~~bash
poetry lock
poetry install
poetry self add poetry-plugin-export
poetry export --without-hashes -f requirements.txt -o requirements.txt
~~~~

Note that `misp-modules` will not be part of the `requirements.txt` file and you will need to create the wheel yourself:

~~~~bash
poetry build --output ./wheels
~~~~

26 changes: 26 additions & 0 deletions documentation/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2821,6 +2821,32 @@ Module to enrich the information by making use of the Vysion API.

-----

#### [Whois Lookup](https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/whois.py)

Module to query a local instance of uwhois (https://github.com/rafiot/uwhoisd).
[[source code](https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/whois.py)]

- **features**:
>This module takes a domain or IP address attribute as input and queries a 'Univseral Whois proxy server' to get the correct details of the Whois query on the input value (check the references for more details about this whois server).

- **config**:
> - server
> - port

- **input**:
>A domain or IP address attribute.

- **output**:
>Text describing the result of a whois request for the input value.

- **references**:
>https://github.com/Lookyloo/uwhoisd

- **requirements**:
>uwhois: A whois python library

-----

#### [WhoisFreaks Lookup](https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/whoisfreaks.py)

<img src=logos/whoisfreaks.png height=60>
Expand Down
114 changes: 114 additions & 0 deletions documentation/mkdocs/expansion.md
Original file line number Diff line number Diff line change
Expand Up @@ -1539,6 +1539,29 @@ Module to access Macvendors API.

-----

#### [Malshare Upload](https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/malshare_upload.py)

Module to push malware samples to MalShare.com
[[source code](https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/malshare_upload.py)]

- **features**:
>The module requires a MalShare API key to upload files, and returns the link of the MalShare analysis.

- **config**:
>api_key

- **input**:
>Attachment or malware sample

- **output**:
>Link attribute that points to the sample at the MalShare analysis instance.

- **references**:
> - https://malshare.com/
> - https://malshare.com/doc.php

-----

#### [Malware Bazaar Lookup](https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/malwarebazaar.py)

Query Malware Bazaar to get additional information about the input hash.
Expand Down Expand Up @@ -2432,6 +2455,42 @@ Module to get information from ThreatMiner.
- **references**:
>https://www.threatminer.org/



-----

#### [Triage Submit](https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/triage_submit.py)

Module to submit samples to tria.ge
[[source code](https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/triage_submit.py)]

- **features**:
> Upload files, and returns the link of the uploaded analysis.
>
>The module can submit URLs to retrieve and analyze them directly in the browser or fetch and execute files in the sandbox.


- **config**:
>apikey
>
>url_mode ( 'submit' or 'fetch' )

- **input**:
>A MISP attribute included in the following list:
>- Attachment
>- malware-sample
>- url

- **output**:
>Link attribute that points to the sample at the Triage analysis instance.

- **references**:
> - https://tria.ge/
> - https://tria.ge/docs/cloud-api/submit/

- **requirements**:
>An access to the Triage API (apikey)

-----

#### [TruSTAR Enrich](https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/trustar_enrich.py)
Expand Down Expand Up @@ -2631,6 +2690,35 @@ Enrich observables with the VirusTotal v3 public API
- **requirements**:
>An access to the VirusTotal API (apikey)


-----

#### [VirusTotal Upload](https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/virustotal_upload.py)

<img src=../logos/virustotal.png height=60>

Module to push malware samples to VirusTotal v3 public API
[[source code](https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/virustotal_upload.py)]

- **features**:
>The module requires a VirusTotal API key to Upload files, and returns the link of the uploaded analysis.

- **config**:
> - apikey

- **input**:
>Attachment or malware sample

- **output**:
>Link attribute that points to the sample at the VirusTotal analysis instance.

- **references**:
> - https://www.virustotal.com
> - https://docs.virustotal.com/reference/overview

- **requirements**:
>An access to the VirusTotal API (apikey)

-----

#### [VMRay Submit](https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/vmray_submit.py)
Expand Down Expand Up @@ -2818,6 +2906,32 @@ Module to enrich the information by making use of the Vysion API.

-----

#### [Whois Lookup](https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/whois.py)

Module to query a local instance of uwhois (https://github.com/rafiot/uwhoisd).
[[source code](https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/whois.py)]

- **features**:
>This module takes a domain or IP address attribute as input and queries a 'Univseral Whois proxy server' to get the correct details of the Whois query on the input value (check the references for more details about this whois server).

- **config**:
> - server
> - port

- **input**:
>A domain or IP address attribute.

- **output**:
>Text describing the result of a whois request for the input value.

- **references**:
>https://github.com/Lookyloo/uwhoisd

- **requirements**:
>uwhois: A whois python library

-----

#### [WhoisFreaks Lookup](https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/whoisfreaks.py)

<img src=../logos/whoisfreaks.png height=60>
Expand Down
1 change: 1 addition & 0 deletions documentation/mkdocs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,7 @@ For more information: [Extending MISP with Python modules](https://www.misp-proj
* [Vulnerability Lookup](https://misp.github.io/misp-modules/expansion/#vulnerability-lookup) - An expansion module to query Vulnerability Lookup
* [Vulners Lookup](https://misp.github.io/misp-modules/expansion/#vulners-lookup) - An expansion hover module to expand information about CVE id using Vulners API.
* [Vysion Enrich](https://misp.github.io/misp-modules/expansion/#vysion-enrich) - Module to enrich the information by making use of the Vysion API.
* [Whois Lookup](https://misp.github.io/misp-modules/expansion/#whois-lookup) - Module to query a local instance of uwhois (https://github.com/rafiot/uwhoisd).
* [WhoisFreaks Lookup](https://misp.github.io/misp-modules/expansion/#whoisfreaks-lookup) - An expansion module for https://whoisfreaks.com/ that will provide an enriched analysis of the provided domain, including WHOIS and DNS information.
* [Wikidata Lookup](https://misp.github.io/misp-modules/expansion/#wikidata-lookup) - An expansion hover module to extract information from Wikidata to have additional information about particular term for analysis.
* [IBM X-Force Exchange Lookup](https://misp.github.io/misp-modules/expansion/#ibm-x-force-exchange-lookup) - An expansion module for IBM X-Force Exchange.
Expand Down
2 changes: 1 addition & 1 deletion misp_modules/modules/expansion/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
'extract_url_components', 'ipinfo', 'whoisfreaks', 'ip2locationio', 'stairwell',
'google_threat_intelligence', 'vulnerability_lookup', 'vysion', 'mcafee_insights_enrich',
'threatfox', 'yeti', 'abuseipdb', 'vmware_nsx', 'sigmf_expand', 'google_safe_browsing',
'google_search', 'whois']
'google_search', 'whois', 'triage_submit', 'virustotal_upload', 'malshare_upload' ]


minimum_required_fields = ('type', 'uuid', 'value')
Expand Down
Loading