Skip to content

Bump main using v2-dev latest #7

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 40 commits into from
Nov 19, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
20e0309
Process Bundle Ui v2 initial commit
clever-age-gtonon Oct 5, 2023
56b0e8d
Add dependency to symfony/runtime
clever-age-gtonon Oct 6, 2023
d7bb55b
Add migration for mysql, postresql & sqlite
clever-age-gtonon Oct 6, 2023
0b945b9
Lint twig templates
clever-age-gtonon Oct 9, 2023
4b22326
Bump actions/checkout from 3 to 4
dependabot[bot] Oct 23, 2023
9db1f59
[Update symfony & easy admin]
clever-age-gtonon Dec 13, 2023
6c005f5
Merge branch 'v2-dev' of github.com:cleverage/processuibundle into v2…
clever-age-gtonon Dec 13, 2023
0adf9a7
[Fix] When multiple messages in queue log filename must be reset befo…
clever-age-gtonon Mar 18, 2024
a42bb49
Add context when run process async
clever-age-gtonon Mar 18, 2024
f2f771a
Update ProcessExecutionStatus.php
xaviermarchegay Jul 23, 2024
7594fda
Update ProcessExecutionStatus.php
xaviermarchegay Jul 23, 2024
6c2bf37
Only display public process (list, execution & logs)
clever-age-gtonon Jul 30, 2024
c3ba778
Add scheduler configuration from UI
clever-age-gtonon Jul 31, 2024
3a768a8
[Feature] Http post request process launcher
clever-age-gtonon Aug 1, 2024
458e6d9
Launch process form to set input && context
clever-age-gtonon Aug 1, 2024
0c6fb77
Fix error on Logs filter
clever-age-gtonon Aug 1, 2024
07d2f26
Add custom filter for log crud on process code or id
clever-age-gtonon Aug 1, 2024
cf86b78
[Fix] Migration for sqlite platform
clever-age-gtonon Aug 26, 2024
1008137
[Fix] Context from form
clever-age-gtonon Aug 28, 2024
992aa1d
[Feature] Add default values && constraints over the execute page
clever-age-gtonon Aug 29, 2024
7dbdb51
[FIX] Error on _em for ProcessExecutionRepository
clever-age-gtonon Sep 3, 2024
b4209ae
Add message to log crud filters
clever-age-gtonon Sep 6, 2024
e116c6c
Fix log handler (log in same file for subprocess) && messenger
clever-age-gtonon Sep 12, 2024
df0e1f7
Remove custom index query builder
clever-age-gtonon Sep 18, 2024
1f37a57
Fix scheduler on first migration run
clever-age-gtonon Sep 18, 2024
15c7b42
Add context to filter on log crud
clever-age-gtonon Sep 24, 2024
aca79fc
Add timezone to user
clever-age-gtonon Oct 7, 2024
252e3ff
Add context to process execution
clever-age-gtonon Oct 7, 2024
374e630
Delete comment on last migration
clever-age-gtonon Oct 8, 2024
c3e6378
Add locale managment to user
clever-age-gtonon Oct 9, 2024
6734e40
PHPCS Fix
clever-age-gtonon Oct 9, 2024
2080fe5
Fix locale set on logout user
clever-age-gtonon Oct 9, 2024
c567e7b
Use default monolog config for process and doctrine handlers
clever-age-gtonon Oct 9, 2024
38d80eb
Fix on scheduler to set context
clever-age-gtonon Oct 16, 2024
d9b127c
Stop workers on scheduler updates
clever-age-gtonon Oct 16, 2024
1ecdfa8
FIX - Process filter on log page
clever-age-gtonon Oct 23, 2024
b0d8999
Fix roles error
clever-age-gtonon Oct 23, 2024
a3d2587
Increase the length of the message of the LogRecord
Oct 24, 2024
ec7a594
Remove deprecated github actions
njoubert-cleverage Nov 19, 2024
6aaf59a
Fix composer.json & remove sqlite support from DoctrineMigrations
njoubert-cleverage Nov 19, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 0 additions & 32 deletions .github/workflows/php.yml

This file was deleted.

19 changes: 0 additions & 19 deletions .github/workflows/super-linter.yml

This file was deleted.

113 changes: 77 additions & 36 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,60 +1,101 @@
![Code Style](https://github.com/cleverage/processuibundle/actions/workflows/super-linter.yml/badge.svg) ![Composer](https://github.com/cleverage/processuibundle/actions/workflows/php.yml/badge.svg)

## CleverAge/ProcessUIBundle
A simple UX for cleverage/processbundle using EasyAdmin

**Installation**
* Import routes
```yaml
#config/routes.yaml
process-ui:
resource: '@CleverAgeProcessUiBundle/Resources/config/routes.yaml'
```
```yaml
processui:
resource: '@CleverAgeProcessUiBundle/src/Controller' type: attribute
```
* Run doctrine migration
* Create an user using cleverage:process-ui:user-create console.

Now you can access Process UI via http://your-domain.com/process

**Indexing logs**
# Features
### Launch process via http request
You can launch a process via http post request
First you need to generate a token via UI User edit form. The ProcessUi generate for you a auth token (keep it in secured area, it will display once).

You can index logs line into database to perform search on ****Process > History**** page.
See configuration section.
It' all, now you can launch a process via http post request

When indexation is enabled you can perform it async.
***Curl sample***
```
curl --location 'https://localhost:8080/http/process/execute?code=demo.die' \
--header 'Authorization: Bearer 3da8409b5f5b640fb0c43d68e8ac8d23' \
--form 'input=@"/file.csv"' \
--form 'context[context_1]="FOO"' \
--form 'context[context_2]="BAR"'
```
* Query string code parameter must be a valid process code
* Header Autorization: Bearer is the previously generated token
* input could be string or file representation
* context you can pass multiple context values

```yaml
#config/messenger.yaml
framework:
messenger:
transports:
log_index: 'doctrine://default'

routing:
CleverAge\ProcessUiBundle\Message\LogIndexerMessage: log_index
```
### Scheduler
You can schedule process execution via UI using cron expression (*/5 * * * *) or periodical triggers (5 seconds)
For more details about cron expression and peridical triggers visit https://symfony.com/doc/6.4/scheduler.html#cron-expression-triggers and https://symfony.com/doc/6.4/scheduler.html#periodical-triggers

Then you have to consume messages by running (use a supervisor to keep consumer alive)
In order to make sheduler process working be sure the following command is running
```
bin/console messenger:consume log_index --memory-limit=64M
bin/console messenger:consume scheduler_cron
```
See more details about ***messenger:consume*** command in consume message section

See official symfony/messenger component documentations for more informations https://symfony.com/doc/current/messenger.html
# Consume Messages
Symfony messenger is used in order to run process via UI or schedule process

**Integrate CrudController**
*To consume process launched via UI make sure the following command is running*
```
bin/console messenger:consume execute_process
```

Of course you can integrate ProcessUI CRUD into your own easy admin Dashboard
```php
public function configureMenuItems(): iterable
{
/* ... your configuration */
yield MenuItem::linkToCrud('History', null, ProcessExecution::class);
}
*To consume scheduled process make sure the following command is running*
```
bin/console messenger:consume scheduler_cron
```
You can pass some options to messenger:consume command
```
Options:
-l, --limit=LIMIT Limit the number of received messages
-f, --failure-limit=FAILURE-LIMIT The number of failed messages the worker can consume
-m, --memory-limit=MEMORY-LIMIT The memory limit the worker can consume
-t, --time-limit=TIME-LIMIT The time limit in seconds the worker can handle new messages
--sleep=SLEEP Seconds to sleep before asking for new messages after no messages were found [default: 1]
-b, --bus=BUS Name of the bus to which received messages should be dispatched (if not passed, bus is determined automatically)
--queues=QUEUES Limit receivers to only consume from the specified queues (multiple values allowed)
--no-reset Do not reset container services after each message
```

**Configuration**
```yaml
clever_age_process_ui:
index_logs:
enabled: false
level: ERROR #Minimum log level to index. Allowed values are DEBUG, INFO, NOTICE, WARNING, ERROR, CRITICAL, ALERT, EMERGENCY
It's recommended to use supervisor app or equivalent to keep command alive

***Sample supervisor configuration***
```
[program:scheduler]
command=php /var/www/html/bin/console messenger:consume scheduler_cron
autostart=false
autorestart=true
startretries=1
startsecs=1
redirect_stderr=true
stderr_logfile=/var/log/supervisor.scheduler-err.log
stdout_logfile=/var/log/supervisor.scheduler-out.log
user=www-data
killasgroup=true
stopasgroup=true

[program:process]
command=php /var/www/html/bin/console messenger:consume execute_process
autostart=false
autorestart=true
startretries=1
startsecs=1
redirect_stderr=true
stderr_logfile=/var/log/supervisor.process-err.log
stdout_logfile=/var/log/supervisor.process-out.log
user=www-data
killasgroup=true
stopasgroup=true
```
136 changes: 51 additions & 85 deletions composer.json
Original file line number Diff line number Diff line change
@@ -1,16 +1,19 @@
{
"name": "cleverage/process-ui-bundle",
"name": "cleverage/ui-process-bundle",
"description": "UI for cleverage/process-bundle",
"keywords": [
"process",
"task",
"etl",
"transformation",
"import",
"export",
"ui"
],
"homepage": "https://github.com/cleverage/ui-process-bundle",
"type": "symfony-bundle",
"license": "MIT",
"description": "UI for cleverage/process-bundle",
"minimum-stability": "dev",
"prefer-stable": true,
"authors": [
{
"name": "Baudouin Douliery",
"email": "[email protected]",
"role": "Developer"
},
{
"name": "Grégory Tonon",
"email": "[email protected]",
Expand All @@ -22,94 +25,57 @@
"role": "Developer"
}
],
"autoload": {
"psr-4": {
"CleverAge\\ProcessUiBundle\\": "src/"
}
},
"autoload-dev": {
"psr-4": {
"CleverAge\\ProcessUiBundle\\Tests\\": "tests/"
}
},
"require": {
"php": ">=8.0",
"php": ">=8.2",
"ext-ctype": "*",
"ext-iconv": "*",
"cleverage/process-bundle": "^3.2",
"easycorp/easyadmin-bundle": "^4.0",
"ddtraceweb/monolog-parser": "^1.3",
"league/flysystem": "^2.2",
"composer/package-versions-deprecated": "^1.11",
"doctrine/doctrine-bundle": "^2.4",
"doctrine/doctrine-migrations-bundle": "^3.1",
"doctrine/orm": "^2.9",
"sensio/framework-extra-bundle": "^6.1",
"symfony/console": "^5.4",
"symfony/messenger": "^5.4",
"symfony/doctrine-messenger": "^5.4",
"symfony/filesystem": "^5.4",
"symfony/flex": "^1.3.1",
"symfony/form": "^5.4",
"symfony/framework-bundle": "^5.4",
"symfony/mime": "^5.4",
"symfony/proxy-manager-bridge": "^5.4",
"symfony/runtime": "^5.4",
"symfony/security-bundle": "^5.4",
"symfony/stopwatch": "^5.4",
"symfony/twig-bundle": "^5.4",
"symfony/validator": "^5.4",
"symfony/webpack-encore-bundle": "^1.11",
"symfony/yaml": "^5.4",
"twig/extra-bundle": "^2.12|^3.0",
"twig/intl-extra": "^3.3"
"ext-pcntl": "*",
"cleverage/process-bundle": "dev-prepare-release",
"doctrine/common": "^3.0",
"doctrine/dbal": "^2.9 || ^3.0",
"doctrine/doctrine-bundle": "^2.5",
"doctrine/doctrine-migrations-bundle": "^3.2",
"doctrine/orm": "^2.9 || ^3.0",
"symfony/dotenv": "^6.4|^7.1",
"symfony/uid": "^6.4|^7.1",
"symfony/string": "^6.4|^7.1",
"symfony/messenger": "^6.4|^7.1",
"symfony/runtime": "^6.4|^7.1",
"symfony/doctrine-messenger": "^6.4|^7.1",
"easycorp/easyadmin-bundle": "^4.8"
},
"require-dev": {
"doctrine/doctrine-fixtures-bundle": "^3.4",
"friendsofphp/php-cs-fixer": "^3.6",
"phpstan/extension-installer": "^1.1",
"phpstan/phpstan": "^1",
"phpstan/phpstan-doctrine": "^1.2",
"phpstan/phpstan-symfony": "^1.1",
"rector/rector": "^0.12.13",
"friendsofphp/php-cs-fixer": "*",
"phpstan/extension-installer": "*",
"phpstan/phpstan": "*",
"phpstan/phpstan-doctrine": "*",
"phpstan/phpstan-symfony": "*",
"phpunit/phpunit": "<10.0",
"rector/rector": "*",
"roave/security-advisories": "dev-latest",
"squizlabs/php_codesniffer": "^3.6",
"symfony/dotenv": "^5.4",
"symfony/browser-kit": "^6.4|^7.1",
"symfony/css-selector": "^6.4|^7.1",
"symfony/debug-bundle": "^6.4|^7.1",
"symfony/maker-bundle": "^1.31",
"symfony/web-profiler-bundle": "^5.4"
"symfony/web-profiler-bundle": "^6.4|^7.1"
},
"config": {
"optimize-autoloader": true,
"preferred-install": {
"*": "dist"
},
"sort-packages": true,
"allow-plugins": {
"composer/package-versions-deprecated": true,
"phpstan/extension-installer": true,
"symfony/flex": true,
"symfony/runtime": true,
"phpstan/extension-installer": true
}
},
"autoload": {
"psr-4": {
"CleverAge\\ProcessUiBundle\\": "src/"
}
},
"autoload-dev": {
"psr-4": {
"CleverAge\\ProcessUiBundle\\Tests\\": "tests/"
}
},
"scripts": {
"auto-scripts": {
"cache:clear": "symfony-cmd",
"assets:install %PUBLIC_DIR%": "symfony-cmd"
"symfony/runtime": true
},
"post-install-cmd": [
"@auto-scripts"
],
"post-update-cmd": [
"@auto-scripts"
]
},
"conflict": {
"symfony/symfony": "*"
},
"extra": {
"symfony": {
"allow-contrib": false,
"require": "5.4.*"
}
"sort-packages": true
}
}
4 changes: 0 additions & 4 deletions config/routes.yaml

This file was deleted.

10 changes: 2 additions & 8 deletions config/services.yaml
Original file line number Diff line number Diff line change
@@ -1,20 +1,14 @@
parameters:
uploads_directory: '%kernel.project_dir%/public/uploads'
cleverage_processes: '%kernel.project_dir%/config/packages'
process_logs_dir: '%kernel.logs_dir%/process'

upload_directory: '%kernel.project_dir%/var/storage/uploads'
services:
_defaults:
autowire: true
autoconfigure: true
bind:
$processLogDir: '%process_logs_dir%'
$indexLogs: '%clever_age_process_ui.index_logs.enabled%'

CleverAge\ProcessUiBundle\:
resource: '../src'
exclude:
- '../src/DependencyInjection/'
- '../src/Entity/'
- '../src/Tests/'
- '../src/Migrations'
- '../src/Migrations'
Binary file added public/logo.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading