Skip to content

#14 Update RemoveFileTask to work with file_pattern or input. Update … #15

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Feb 25, 2025
Merged

#14 Update RemoveFileTask to work with file_pattern or input. Update … #15

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions docs/reference/tasks/01-FileFetchTask.md
View file
Original file line number Diff line number Diff line change
@@ -17,7 +17,7 @@ If the option `file_pattern` is not set the input is used as strict filename(s)

If input is set but not corresponding as any file into `source_filesystem` task failed with UnableToReadFile exception.

If FileFetchTask is the fisrt task of you process and you wan to use input, don't forgive to set the `entry_point` task name at process level
If FileFetchTask is the first task of you process and you want to use input, don't forgive to set the `entry_point` task name at process level

Possible outputs
----------------
@@ -27,12 +27,12 @@ Filename of copied file.
Options
-------

| Code | Type | Required | Default | Description |
|--------------------------|:----------:|:---------:|:---------:|-----------------------------------------------------------------------------------------------------------------------------------------------|
| `source_filesystem` | `string` | **X** | | The source flysystem/storage.<br/>See config/packages/flysystem.yaml to see configured flysystem/storages. |
| `destination_filesystem` | `string` | **X** | | The source flysystem/storage.<br/>See config/packages/flysystem.yaml to see configured flysystem/storages. |
| `file_pattern` | `string` | | null | The file_parttern used in preg_match to match into `source_filesystem` list of files. If not set try to use input as strict filename to match |
| `remove_source` | `bool` | | false | If true delete source file after copy |
| Code | Type | Required | Default | Description |
|--------------------------|:----------:|:---------:|:---------:|----------------------------------------------------------------------------------------------------------------------------------------------|
| `source_filesystem` | `string` | **X** | | The source flysystem/storage.<br/>See config/packages/flysystem.yaml to see configured flysystem/storages. |
| `destination_filesystem` | `string` | **X** | | The source flysystem/storage.<br/>See config/packages/flysystem.yaml to see configured flysystem/storages. |
| `file_pattern` | `string` | | null | The file_pattern used in preg_match to match into `source_filesystem` list of files. If not set try to use input as strict filename to match |
| `remove_source` | `bool` | | false | If true delete source file after copy |


Examples
30 changes: 24 additions & 6 deletions docs/reference/tasks/03-RemoveFileTask.md
View file
Original file line number Diff line number Diff line change
@@ -13,6 +13,8 @@ Accepted inputs

The filename of the file to remove on `filesystem`.

If the option `file_pattern` is not set the input is used as strict filename(s) to match.

When filename is deleted add a info log.

If filename not found or cannot be deleted on `filesystem` add a warning log.
@@ -25,22 +27,38 @@ None
Options
-------

| Code | Type | Required | Default | Description |
|--------------|:----------:|:----------:|:---------:|-------------------------------------------------------------------------------------------------------------|
| `filesystem` | `string` | **X** | | The source flysystem/storage. <br/>See config/packages/flysystem.yaml to see configured flysystem/storages. |
| Code | Type | Required | Default | Description |
|----------------|:----------:|:--------:|:---------:|----------------------------------------------------------------------------------------------------------------------------------------------|
| `filesystem` | `string` | **X** | | The source flysystem/storage. <br/>See config/packages/flysystem.yaml to see configured flysystem/storages. |
| `file_pattern` | `string` | | null | The file_pattern used in preg_match to match into `source_filesystem` list of files. If not set try to use input as strict filename to match |


Examples
--------
* Simple process to remove a file on 'storage.source' via <br>```bin/console cleverage:process:execute my_custom_process --input=foobar.csv -vv```
* Simple process to remove a file on 'storage.source' via <br>```bin/console cleverage:process:execute input_process --input=foobar.csv -vv```
- see config/packages/flysystem.yaml to see configured flysystems/storages.
- remove file with name passed as input
```yaml
#
my_custom_process:
input_process:
entry_point: remove_from_input
tasks:
remove_from_input:
service: '@CleverAge\FlysystemProcessBundle\Task\FileFetchTask'
service: '@CleverAge\FlysystemProcessBundle\Task\RemoveFileTask'
options:
filesystem: 'storage.source'
```

* Simple process to remove a file on 'storage.source' via <br>```bin/console cleverage:process:execute pattern_process -vv```
- see config/packages/flysystem.yaml to see configured flysystems/storages.
- remove files filtered by file_pattern
```yaml
#
pattern_process:
tasks:
remove_from_file_pattern:
service: '@CleverAge\FlysystemProcessBundle\Task\RemoveFileTask'
options:
filesystem: 'storage.source'
file_pattern: '/.csv$/'
```
34 changes: 29 additions & 5 deletions src/Task/RemoveFileTask.php
View file
Original file line number Diff line number Diff line change
@@ -22,10 +22,12 @@
use Symfony\Component\OptionsResolver\OptionsResolver;

/**
* Remove a file from a filesystem.
* Remove files from a filesystem.
*/
class RemoveFileTask extends AbstractConfigurableTask
{
private FilesystemOperator $filesystem;

/**
* @param ServiceLocator<FilesystemOperator> $storages
*/
@@ -37,18 +39,40 @@ protected function configureOptions(OptionsResolver $resolver): void
{
$resolver->setRequired('filesystem');
$resolver->setAllowedTypes('filesystem', 'string');

$resolver->setDefault('file_pattern', null);
$resolver->setAllowedTypes('file_pattern', ['string', 'null']);
}

public function execute(ProcessState $state): void
{
/** @var string $filesystemOption */
$filesystemOption = $this->getOption($state, 'filesystem');
$filesystem = $this->storages->get($filesystemOption);
/** @var string $filePath */
$filePath = $state->getInput();
$this->filesystem = $this->storages->get($filesystemOption);

/** @var ?string $filePattern */
$filePattern = $this->getOption($state, 'file_pattern');
if ($filePattern) {
foreach ($this->filesystem->listContents('/') as $file) {
if ('file' === $file->type() && preg_match($filePattern, $file->path())) {
$this->deleteFile($file->path());
}
}
} else {
/** @var ?string $input */
$input = $state->getInput();
if (!$input) {
throw new \UnexpectedValueException('No pattern neither input provided for the Task');
}

$this->deleteFile($input);
}
}

private function deleteFile(string $filePath): void
{
try {
$filesystem->delete($filePath);
$this->filesystem->delete($filePath);
$result = true;
} catch (FilesystemException) {
$result = false;
Loading