Free concurrency mode

[Harfusha]

Would be nice to have a 3rd option to allow the same task to deploy concurrently. I've been using semaphore to provision VM's on my proxmox cluster.

There's no reason I couldn't run the same task concurrently in this scenario and it would have been nice if the option was there.

Originally posted by @jfloodnet in #366 (comment)

The tasks we are running cannot collide with each other so free strategy would be nice to have.

[TallerAxouxere]

idem, i need 5 task with the same playbook :(

[rajcz]

Hello all,

I also require this. I'm managing over ten tasks within a single playbook simultaneously :). Does anyone on the team have an idea of when enhancements might be implemented? Thank you!

Pavel

[fiftin]

Hi @tboerger do you have a time to do this? :)

[Harfusha]

Hello, is there any update on this issue?

[fatihusta]

Hi @Harfusha

If I understand correctly, the ansible configuration below may help you.

Create this file in the project folder.
ansible.cfg

Copy these config into ansible.cfg

[defaults]
force_color = True
host_key_checking = False
callbacks_enabled = profile_tasks, timer
interpreter_python = auto

[ssh_connection]
ssh_args = -o ControlMaster=auto -o ControlPersist=30m -o ConnectionAttempts=100 -o UserKnownHostsFile=/dev/null
pipelining = True

The important part is ssh_connection section.

This configuration runs the same task in parallel on different hosts.

[Harfusha]

Hello, but we need to run multiple tasks at same time at same host. We are running tasks on same server but for diffrent client domains.

[fatihusta]

Ansible has async feature, but I never tried.

[Harfusha]

But the tasks we are running are independent. There can be running 5 times 1 playbook with diffrent parameters on single host and 5 diffrent playbooks on same host at a same time.

When this happens, than the last task can be waiting more than hour.

We cannot group them because each of them is doing something else or has diffrent target. And they are not ran at a same timre but when multiple long tasks are added, than the queue is created.

We have our task created that all of them are independent and cannot collide with any other task.

We are using parallel tasks for all we can, but this is not possible when the tasks are added at diffrent time.

If free concurrency mode would be added task would not wait for each other. When i add up time from all tasks then 80% of time is just waiting for task to finish.

[jfloodnet]

Yep, the use case for me is a single target host that can handle the same task executing concurrently. The current model doesn't allow it.

[TallerAxouxere]

I want to launch the same playbook on multiple machines simultaneously, but with the adjustment you indicate, I couldn't achieve it. Use different IPs on each launch.

[Harfusha]

Any update on this issue?

[Harfusha]

Any updates?

[Harfusha]

Any updates?

[MarcoRizk]

This feature would make the tool more usable, since most of the time we need to run the same playbook multiple times on different hosts and i don't want to be blocked when the need emerges to run on another host while another is running knowing that some of our tasks take a long time to run

[jdhayes]

A similar discussion we had previously here.

[gaoyue1989]

I think we can add a field “allow_simultaneous” similar to awx when setting the template。
In order to avoid some conflict problems

A similar discussion we had previously here.

[MarcoRizk]

I was gonna suggest something similar to @gaoyue1989 a boolean flag in the task template should allow users to customize how they want their tasks to run

[gaoyue1989]

From the previous discussion, the previous conflict may be caused by executing git clone for each task. Considering that git changes are not frequent, you can add memory cache during execution. For example, if the task is running, execute git clone at intervals. If it is within the cache time, use the local directory file

A similar discussion we had previously here.

[Harfusha]

I see you have open PR for non git based repos, but i need this functionality for git repos. Why you dont add condition if last commit (That would normally be cloned) is already cloned in current directory, that way both can run side by side. If commit would be diffrent, task will normally wait.