You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
First of all, thank you for choosing to contribute to our project. This short guide will describe the required coding conventions, as well as the built-in tests that we use in our project. Following these guidelines will help us merge your pull request into our code base in a fast as smooth manner.
4
+
5
+
Submitting changes
6
+
------------------
7
+
Please send us a GitHub Pull Request with a clear list of what you've done. Make sure to include a clear log message for your commits, describing the modifications / additions to the code base, and their implications.
8
+
9
+
Reporting a bug
10
+
---------------
11
+
In order to help us close the bug as quickly as possible, please follow these steps:
12
+
1. Make sure there isn't an open issue that already addresses this bug
13
+
2. If there isn't, open a new issue and attach as many informative details as you can:
14
+
* Architecture & compilation flags in which the bug occurs
15
+
* Trace + logs, describing the bug / exception
16
+
* As many details as you can in order to help us reproduce and fix this issue
17
+
3. If you already have a fix, please submit it as a pull request, and include the bug details in it's description
18
+
19
+
Coding Conventions
20
+
------------------
21
+
Start reading our code and we believe you'll get the hang of it.
22
+
The important notes are:
23
+
* Every function should be documented in a manner that is consistent with the current documentation standard
24
+
* Each indentation level should be 4 spaces in width (spaces, not tabs)
25
+
* We believe that comments improve the readability, make sure that your code is documented enough to be understood by other developers
26
+
27
+
Testing
28
+
-------
29
+
**Scout** uses the following tools to enforce coding conventions and to help eliminate common python bugs:
Testing can be done from the project's home directory:
34
+
1. Testing for **pydocstyle**:
35
+
```Scout> python tests.py```
36
+
2. Testing for **flake8**:
37
+
```Scout> flake8 src```
38
+
39
+
One last note
40
+
-------------
41
+
We believe that the only way that Open Source tools will help the infosec community in the long term, is to maintain these tools and to make sure they are developed according to the community standards. Each contribution brings as one step further to this goal.
Copy file name to clipboardExpand all lines: docs/Adding Custom Instructions.md
+10-9Lines changed: 10 additions & 9 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,9 +4,9 @@ Being an instruction-based debugger, Scout supports project extensions.
4
4
5
5
Registration - C Code
6
6
---------------------
7
-
* Each of the instructions that are added by the project, should be registerred by calling ```register_instruction()```.
8
-
* The registration should take place inside the function ```register_specific_instructions()```.
9
-
* This design makes sure that when invoking ```register_all_instructions()```, all of the default instructions, and extension instruction, will be registerred correctly.
7
+
* Each of the instructions that are added by the project, should be registerred by calling ``register_instruction()``.
8
+
* The registration should take place inside the function ``register_specific_instructions()``.
9
+
* This design makes sure that when invoking ``register_all_instructions()``, all of the default instructions, and extension instruction, will be registerred correctly.
10
10
11
11
Implementation - C Code
12
12
-----------------------
@@ -15,15 +15,15 @@ In order to implement a new instruction, one should define each of the required
15
15
* Instruction ID - must be unique, but not necessarily consecutive
16
16
* Minimal Length - minimal amount of bytes needed for a valid instruction (robustness checks)
17
17
* Maximal Length - maximal amount of bytes needed for a valid instruction (robustness checks)
18
-
* Instruction handler - a handler function with a fixed signature of: ```int32_t (*instrHandler)(void * ctx, uint8_t * instruction, uint32_t length)```
18
+
* Instruction handler - a handler function with a fixed signature of: ``int32_t (*instrHandler)(void * ctx, uint8_t * instruction, uint32_t length)``
19
19
20
-
**Note:** The instructions are stored in a global array with a **fixed** capacity. When adding new instructions, one should make sure to adjust this capacity accordingly.
21
-
The capacity is defined in ```scout_api.h``` and is set by default to ```#define SCOUT_MAX_INSTRS (10)```.
20
+
**Note:** The instructions are stored in a global array with a **fixed** capacity. When adding new instructions, one should make sure to adjust this capacity accordingly (both in the C and .py files).
21
+
The capacity is defined in ``scout_api.h`` and is set by default to ``#define SCOUT_MAX_INSTRS (10)``.
22
22
23
23
Examples - C Code
24
24
-----------------
25
-
* Embedded mode (```embedded_scout```) - files ```project_instructions``` (*.c and *.h)
26
-
* Linux Kernel mode (```kernel_scout```) - files ```driver\scout_kernel_instructions``` (*.c and *.h)
25
+
* Embedded mode (``embedded_scout``) - files ``project_instructions`` (*.c and *.h)
26
+
* Linux Kernel mode (``kernel_scout``) - files ``driver\scout_kernel_instructions`` (*.c and *.h)
27
27
28
28
Client Side - Python Code
29
29
-------------------------
@@ -34,4 +34,5 @@ In the client side, adding a new instructions is even easier, and requires only
34
34
35
35
Examples - Python Code
36
36
----------------------
37
-
* Linux Kernel example (```manager```) - file ```kernel_scout_api.py```
37
+
* Embedded example (``manager``) - file ``embedded_scout_api.py``
38
+
* Linux Kernel example (``manager``) - file ``kernel_scout_api.py``
To decide what will be the suitable compilation mode / architecture flags, one should check the following parameters.
11
12
Each of the defined parameters is a C MACRO (define) that controls the behavior (and compilation) of the resulting binary.
12
13
14
+
**Important Note:**
15
+
When using ``scoutCompiler``, it will automatically generate most of the needed flags by deducing the right values based on the architecture and configuration flags that are supplied to it. Please see the ``embedded_scout`` example for more info.
16
+
13
17
Target Endianness
14
18
-----------------
15
19
* SCOUT_BIG_ENDIAN - Scout is executed on a Big Endian architecture
@@ -36,48 +40,51 @@ Only one of above flags can be defined.
36
40
If none are defined the base library will define "SCOUT_ARCH_INTEL" on it's own.
37
41
38
42
**Additional Flags:**
39
-
*SCOUT_ARM_THUMB - Scout will be executed on an ARM cpu in Thumb mode. Can only be used together with the "SCOUT_ARCH_ARM" flag.
43
+
SCOUT_ARM_THUMB - Scout will be executed on an ARM cpu in Thumb mode. Can only be used together with the "SCOUT_ARCH_ARM" flag.
40
44
41
-
The flags is needed only in PIC mode, in which we use inline assembly.
45
+
The flags are needed only in PIC mode, in which we use inline assembly.
42
46
43
47
Target Permission Level
44
48
-----------------------
45
-
* SCOUT_MODE_USER - Scout is executed in User-Mode
46
-
* SCOUT_MODE_KERNEL - Scout is executed in Kernel-Mode
49
+
* SCOUT_MODE_USER - Scout is executed in User-Mode (& low CPU privileges)
50
+
* SCOUT_MODE_KERNEL - Scout is executed in Kernel-Mode (& high CPU privileges)
47
51
48
52
Only one of above flags can be defined.
49
53
If none are defined the base library will define "SCOUT_MODE_USER" on it's own.
50
54
51
-
**Note:** These flags are used only in a Linux PC Environment, and are not used in an Embedded Environment.
52
-
53
-
Target Loading Environment
54
-
--------------------------
55
-
* SCOUT_PC_ENV - Scout is executed as a standard process (user) or driver (kernel) on a Linux machine
56
-
* SCOUT_EMBEDDED_ENV - Scout is injected to the address space of a given executable
57
-
58
-
Only one of above flags can be defined.
59
-
If none are defined the base library will define "SCOUT_PC_ENV" on it's own.
60
-
61
-
**Note:** SCOUT_EMBEDDED_ENV has many use cases, including:
62
-
1. Injecting a debugger into a debuggee Linux process
63
-
2. Injecting a debugger into a debuggee firmware (if the executable's API matches the basic POSIX based API of Scout)
55
+
"SCOUT_MODE_KERNEL" will also be the right choice for an RTOS (Real-time OS) in which every task / our task has high privileges. The flag will lead to the definition of "SCOUT_HIGH_PRIVILEGES" by the compilation environment.
64
56
65
-
**Note:**At the current moment, "SCOUT_EMBEDDED_ENV" must be used with "SCOUT_PIC_CODE", although in the future a linker script could help an embedded scout access external functions without the PIC context.
57
+
**Important Note:**As flushing the CPU cache usually requires high privileges, the per-architecture implementation will only be available if compiling using the "SCOUT_MODE_KERNEL" flag.
66
58
67
59
Position Independent Mode - SCOUT_PIC_CODE
68
60
------------------------------------------
69
61
Scout will be compiled for full Position Independent Code (PIC) mode. Any access to an external function / global variable will pass through a unique "Context" object. Read the section about "PIC Compilation" for more information.
70
-
**Note:** Can only be used with "SCOUT_EMBEDDED_ENV".
62
+
63
+
"SCOUT_PIC_CODE" will lead to the definition of "SCOUT_ISOLATED_ENV" by the compilation environment, because a PIC blob will always be isolated from the environment, and won't have the luxory of a proper executable loader such as "ld.so".
64
+
65
+
Host LibC Implementation
66
+
------------------------
67
+
When injecting our (PIC) code into a host binary, we should make sure to use the proper constants for the matching standard library implementation that is used by the respective binary:
68
+
* SCOUT_HOST_GLIBC - The used library is Glibc
69
+
* SCOUT_HOST_UCLIBC - The used library is uClibc (or uClibc-NG)
70
+
71
+
If none are defined the base library will define "SCOUT_HOST_GLIBC" on it's own.
71
72
72
73
Loader Flags
73
74
------------
74
75
* SCOUT_LOADER - We are now compiling a loader (that might be using it's own pic plt / globals).
75
76
* SCOUT_LOADING_THUMB_CODE - The loader will load a Scout that was compiled to be executed on an ARM cpu in Thumb mode.
76
-
* SCOUT_RESTORE_FLOW - The default loaders (```tcp_client_server.c```, ```tcp_loader_server.c```) will clean-up after themselves if the loaded scout will finish his endless loop.
77
+
* SCOUT_RESTORE_FLOW - The default loaders (``tcp_client_server.c``, ``tcp_loader_server.c``) will clean-up after themselves if the loaded scout will finish the endless loop.
78
+
79
+
If the loader will be compiled to be Position Independent (PIC), which is probably the most common use case, it will also define a new flag of "SCOUT_SLIM_SIZE", to help shrink the size of the binary (to serve as an effective shellcode).
80
+
Under this definition the TCP server would expect the following flags (if needed):
81
+
* SCOUT_TCP_CLIENT - There is a need to include the feature of a TCP client
82
+
* SCOUT_TCP_SERVER - There is a need to include the feature of a TCP server
83
+
* SCOUT_TCP_SEND - There is a need to include the ability to reliably send TCP messages
77
84
78
85
Additional Flags:
79
86
-----------------
80
87
* SCOUT_INSTRUCTIONS - Scout is going to use the instructions api (using the TCP server for instance)
81
-
* SCOUT_DYNAMIC_BUFFERS - Scout will dynamically malloc() buffers to be used by the tcp server. Otherwise static buffers will be used.
88
+
* SCOUT_DYNAMIC_BUFFERS - Scout will dynamically ``malloc()`` buffers to be used by the tcp server. Otherwise static buffers will be used.
82
89
* SCOUT_PROXY - Scout is going to act as a proxy (user scout passing instructions to a kernel driver for instance)
83
-
* SCOUT_MMAP - Should scout's loaders use mmap() and mprotect() when loading (if defined) or should they simply use malloc (if undefined)
90
+
* SCOUT_MMAP - Should scout's loaders use ``mmap()`` and ``mprotect()`` when loading (if defined) or should they simply use ``malloc()`` (if undefined)
Copy file name to clipboardExpand all lines: docs/Default Instructions.md
+4-3Lines changed: 4 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,19 +4,20 @@ Scout is an instruction-based debugger, that commonly uses a TCP network session
4
4
5
5
Default Instructions
6
6
--------------------
7
-
***NOP** - Used as a Pong (or Keep-Alive) instruction to make sure the debugger is active and responds to commands
7
+
***NOP** - Used as a Ping (or Keep-Alive) instruction to make sure the debugger is active and responds to commands
8
8
***Memory Read** - Reads (virtual) memory from the given address, and sends it back
9
9
***Memory Write** - Writes a given binary content to a (virtual) memory in the debuggee's address space
10
10
11
-
Each supported instruction must be pre-registered by the debugger before it enters his server loop, usually by calling ```register_all_instructions()```.
11
+
Each supported instruction must be pre-registered by the debugger before it enters his server loop, usually by calling ``register_all_instructions()``.
12
12
13
13
Network API
14
14
-----------
15
15
Each instruction is sent together with a network header that includes the following:
16
+
16
17
* Instruction ID - 2 Bytes
17
18
* Length field - 4 Bytes
18
19
19
20
The length field specifies the length, in bytes, of the serialized instruction.
20
21
21
22
**Note:** All instructions should be serialized to NETWORK order.
22
-
See ```manager\scout_api.py``` for a python sample that prepares the instructions for network transmission.
23
+
See ``manager\scout_api.py`` for a python sample that prepares the instructions for network transmission.
Copy file name to clipboardExpand all lines: docs/Default Loaders.md
+7-7Lines changed: 7 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,8 +8,8 @@ To overcome these limitations, the scout debugger comes with a list of supported
8
8
9
9
The defualt loaders are:
10
10
11
-
1.```tcp_client_loader.c``` - Connects to a predefined TCP server
12
-
2.```tcp_server_loader.c``` - Waits for an incoming TCP client
11
+
1.``tcp_client_loader.c`` - Connects to a predefined TCP server
12
+
2.``tcp_server_loader.c`` - Waits for an incoming TCP client
13
13
14
14
Both of the loaders use the same network protocol:
15
15
@@ -18,13 +18,13 @@ Both of the loaders use the same network protocol:
18
18
19
19
After a TCP connection was established, the loader will follow these steps:
20
20
21
-
1. The loader will receive the header and malloc() a memory buffer of appropriate size.
21
+
1. The loader will receive the header and ``malloc()``/``mmap()`` a memory buffer of appropriate size.
22
22
2. The data will be received and stored in this memory buffer.
23
-
3. The loader will flush the D-cache and I-cache of the buffer (only in architectures were this is needed)
24
-
4. The loader will mprotect() the memory to use the correct permissions (only in architectures were this is needed)
23
+
3. The loader will flush the D-cache and I-cache of the buffer (only in architectures were this is needed, and only if we have high enough CPU privileges for it)
24
+
4. The loader will ``mprotect()`` the memory to use the correct permissions (only in architectures were this is needed)
25
25
5. The loader will jump into the buffer's start (offset 0, as mentioned in the "PIC Compilation" section)
26
-
6. In the flow restore case, once the loaded executable finishes the loader will free the memory and close the used sockets.
26
+
6. In the flow restore case, once the loaded executable finishes, the loader will free the memory and close the used sockets.
27
27
28
-
The functions ```remoteLoadServer()``` and ```remoteLoadClient()``` in ```scout_network.py```, implement the required protocol for communicating with the loader, and loading up the full Scout.
28
+
The functions ``remoteLoadServer()`` and ``remoteLoadClient()`` in ``scout_network.py``, implement the required protocol for communicating with the loader, and loading up the full Scout. A working example is shown under the ``manager.py`` of the ``embedded_scout`` example.
29
29
30
30
**Note:** In case there are any W^X style limitations in your environment, it is recommended to make sure that the allocated memory for the full debugger will have both Write and eXcutable permissions (should probably use the SCOUT_MMAP flag in this case).
0 commit comments