Initial Commit

Author: ltd-ARYAN-pvt

1) Multithreading/README.md

@@ -0,0 +1,111 @@
+# Multithreading in Python: A Simple Guide
+
+## What is Multithreading?
+
+Multithreading allows you to run multiple parts of your program simultaneously. In Python, this means you can have different tasks running at the same time. Multithreading is particularly useful when you have IO-bound tasks like reading/writing files or making network requests. However, for CPU-bound tasks, multithreading in Python might not provide much benefit due to the Global Interpreter Lock (GIL).
+
+## Why Use Multithreading?
+
+- **Faster Execution**: For tasks that can run concurrently (like waiting for a file to load), multithreading can improve the program's speed.
+- **Better Resource Utilization**: Multiple threads can use system resources efficiently, allowing your program to do multiple things at once.
+  
+### Key Libraries:
+- `threading`: Provides low-level thread control.
+- `concurrent.futures`: A high-level interface for running tasks concurrently.
+
+---
+
+## Using `threading` in Python
+
+The `threading` library allows you to create and manage threads. Each thread runs a function concurrently.
+
+### Example 1: Using `threading`
+
+```python
+import threading
+import time
+
+def print_numbers():
+    for i in range(5):
+        print(f"Number: {i}")
+        time.sleep(1)
+
+def print_letters():
+    for letter in "ABCDE":
+        print(f"Letter: {letter}")
+        time.sleep(1)
+
+# Create threads
+thread1 = threading.Thread(target=print_numbers)
+thread2 = threading.Thread(target=print_letters)
+
+# Start threads
+thread1.start()
+thread2.start()
+
+# Wait for both threads to finish
+thread1.join()
+thread2.join()
+
+print("Both threads have finished execution.")
+```
+
+### Explanation:
+- `threading.Thread`: Creates a new thread that runs a specified function.
+- `.start()`: Starts the thread.
+- `.join()`: Waits for the thread to complete before continuing.
+
+In this example, the two threads will print numbers and letters concurrently.
+
+---
+
+## Using `concurrent.futures` for Multithreading
+
+The `concurrent.futures` library provides a simpler and higher-level interface for handling threading and multiprocessing. It uses thread pools to manage multiple threads.
+
+### Example 2: Using `concurrent.futures.ThreadPoolExecutor`
+
+```python
+from concurrent.futures import ThreadPoolExecutor
+import time
+
+def task(name, duration):
+    print(f"Task {name} starting.")
+    time.sleep(duration)
+    print(f"Task {name} completed after {duration} seconds.")
+
+# Create a ThreadPoolExecutor
+with ThreadPoolExecutor(max_workers=2) as executor:
+    executor.submit(task, 'A', 2)
+    executor.submit(task, 'B', 3)
+
+print("All tasks are submitted.")
+```
+
+### Explanation:
+- `ThreadPoolExecutor`: Manages a pool of worker threads that execute tasks concurrently.
+- `.submit()`: Schedules a function to be executed by the thread pool.
+- `max_workers`: Defines the maximum number of threads that can run at the same time.
+
+In this example, the `ThreadPoolExecutor` runs two tasks (`task 'A'` and `task 'B'`) concurrently.
+
+---
+
+## When to Use `threading` vs `concurrent.futures`?
+
+- Use `threading` when you need more control over each thread (e.g., starting, stopping, or managing multiple threads manually).
+- Use `concurrent.futures` when you want a simpler, high-level interface for managing multiple tasks concurrently.
+
+---
+
+## Key Considerations:
+- **Global Interpreter Lock (GIL)**: Python threads are limited by the GIL, meaning only one thread executes Python bytecode at a time. For CPU-bound tasks, consider using `concurrent.futures.ProcessPoolExecutor` or multiprocessing.
+- **Use Cases**: Multithreading is great for IO-bound tasks like downloading files, web scraping, or handling database connections.
+
+---
+
+### Further Reading:
+- [Python threading documentation](https://docs.python.org/3/library/threading.html)
+- [concurrent.futures documentation](https://docs.python.org/3/library/concurrent.futures.html)
+
+Note:- It is rephrased by ChatGPT 4o

1) Multithreading/basics/1)_threading_basic.py

@@ -0,0 +1,36 @@
+import threading
+import time
+
+def let_sleep(sec):
+    print(f"Program is sleep for {sec} seconds")
+    time.sleep(sec)
+    print(f"Program waked up after {sec} seconds")
+
+start_t=time.time()
+let_sleep(4)
+let_sleep(3)
+let_sleep(2)
+end_t=time.time()
+print(f"Time taken to complete above process is {end_t-start_t} sec")
+# -> This will take nearly 9 secs to complete
+
+
+start_t=time.time()
+t1=threading.Thread(target=let_sleep, args=[4])
+t2=threading.Thread(target=let_sleep, args=[3])
+t3=threading.Thread(target=let_sleep, args=[2])
+
+#--> t.start():- this will start the process and send it to the backend
+
+t1.start()
+t2.start()
+t3.start()
+
+#--> t.join():- This will tell that wait for threads to complete it's execution
+
+t1.join()
+t2.join()
+t3.join()
+end_t=time.time()
+
+print(f"Time taken to complete above process is {end_t-start_t} sec")

1) Multithreading/basics/2)_threadpool_executor.py

@@ -0,0 +1,32 @@
+from concurrent.futures import ThreadPoolExecutor
+import time
+
+def let_sleep(sec):
+    print(f"Program is sleep for {sec} seconds")
+    time.sleep(sec)
+    print(f"Program waked up after {sec} seconds")
+
+#--> Method 1 :- Creating individual threads in threadpool
+
+def pooling():
+    with ThreadPoolExecutor(max_workers=4) as executor:
+        f1=executor.submit(let_sleep,4)
+        f2=executor.submit(let_sleep,3)
+        f3=executor.submit(let_sleep,2)
+        f4=executor.submit(let_sleep,3)
+        print(f1.result())
+        print(f2.result())
+        print(f3.result())
+        print(f4.result())
+
+
+#--> Method 2 :- Using map or iterator
+
+def pooling_map():
+    with ThreadPoolExecutor(max_workers=4) as executor:
+        ls=[4,3,2,3]
+        results=executor.map(let_sleep,ls)
+        for r in results:
+            print(r)
+
+pooling_map()

1) Multithreading/project/Image_downloader_app.py

@@ -0,0 +1,56 @@
+from concurrent.futures import ThreadPoolExecutor
+import os
+import requests
+import time
+
+img_file_seq='img_src_seq'
+img_file_mt='img_src_mt'
+
+#--> small height and width img
+# url='https://picsum.photos/200/300' 
+
+#--> big height and width img
+url='https://picsum.photos/2000/3000'
+
+def create_folder(name):
+    if not os.path.exists(name):
+        os.makedirs(name)
+    return name
+
+def download_img(links, file_name):
+    response=requests.get(links)
+    if response.status_code == 200:
+        with open(file_name,'wb') as f:
+            f.write(response.content)
+        print(f"{os.path.basename(file_name)} Downloaded! ")
+    else:
+        print(f"Failed to download the {file_name}. Status code:- {response.status_code}")
+    print()
+
+def download_img_seq(img_file, img_num):
+    folder=create_folder(img_file)
+
+    for i in range(img_num):
+        image_name=f"img-{i+1}.jpg"
+        file_name=os.path.join(folder,image_name)
+        download_img(url,file_name)
+
+def download_img_multithreading(img_file, img_num, num_threads=5):
+    folder=create_folder(img_file)
+
+    with ThreadPoolExecutor(max_workers=num_threads) as executor:
+        futures = [executor.submit(download_img, url, os.path.join(folder, f"image_threaded_{i + 1}.jpg")) for i in range(img_num)]
+        # for future in futures:
+        #     print(future.result())
+
+def func_run(func, img_file, img_num, tag):
+    start_t=time.time()
+    func(img_file, img_num)
+    total_t=time.time()-start_t
+    print(f"Total time take to download {img_num} images is {total_t: .2f} seconds for {tag}")
+
+if __name__ == "__main__":
+    img_num=5
+    func_run(download_img_seq,img_file_seq,img_num, "sequential method")
+
+    func_run(download_img_multithreading,img_file_mt,img_num, "multithreading method")

2) Multicore Processing/README.md

@@ -0,0 +1,115 @@
+# Multicore Processing in Python: A Simple Guide
+
+## What is Multicore Processing?
+
+Multicore processing allows programs to run on multiple CPU cores simultaneously. This is especially useful for CPU-bound tasks that require a lot of computation, like data processing, mathematical calculations, or machine learning.
+
+Unlike multithreading (which is limited by Python’s Global Interpreter Lock or GIL), multiprocessing runs separate processes, each with its own Python interpreter, which allows full utilization of multiple cores.
+
+## Why Use Multicore Processing?
+
+- **Parallelism**: Multicore processing allows for true parallel execution, speeding up CPU-intensive tasks.
+- **No GIL Limitation**: Since each process has its own Python interpreter, it avoids the GIL, allowing multiple cores to run Python code in parallel.
+
+### Key Libraries:
+- `multiprocessing`: Provides tools for spawning multiple processes and managing shared data between them.
+- `concurrent.futures`: A high-level interface that can also be used for multiprocessing via `ProcessPoolExecutor`.
+
+---
+
+## Using `multiprocessing` in Python
+
+The `multiprocessing` library allows you to create and manage multiple processes. Each process runs independently, and they can run on different CPU cores.
+
+### Example 1: Using `multiprocessing`
+
+```python
+import multiprocessing
+import time
+
+def square_number(number):
+    print(f"Square of {number} is {number * number}")
+    time.sleep(1)
+
+if __name__ == "__main__":
+    processes = []
+    numbers = [1, 2, 3, 4, 5]
+
+    # Create a process for each number
+    for number in numbers:
+        process = multiprocessing.Process(target=square_number, args=(number,))
+        processes.append(process)
+        process.start()
+
+    # Wait for all processes to complete
+    for process in processes:
+        process.join()
+
+    print("All processes have finished execution.")
+```
+
+### Explanation:
+- `multiprocessing.Process`: Creates a new process.
+- `.start()`: Starts the process.
+- `.join()`: Waits for the process to complete before moving on.
+- `if __name__ == "__main__"`: Ensures that the code block is only run in the main process (important for Windows systems).
+
+In this example, each number is processed by a separate CPU core, and the squares are calculated in parallel.
+
+---
+
+## Using `concurrent.futures` for Multiprocessing
+
+The `concurrent.futures` library can also be used for multiprocessing through `ProcessPoolExecutor`. This provides a simpler, high-level API for multicore processing.
+
+### Example 2: Using `concurrent.futures.ProcessPoolExecutor`
+
+```python
+from concurrent.futures import ProcessPoolExecutor
+import time
+
+def cube_number(number):
+    print(f"Cube of {number} is {number * number * number}")
+    time.sleep(1)
+
+numbers = [1, 2, 3, 4, 5]
+
+# Create a ProcessPoolExecutor
+with ProcessPoolExecutor() as executor:
+    executor.map(cube_number, numbers)
+
+print("All tasks are completed.")
+```
+
+### Explanation:
+- `ProcessPoolExecutor`: Manages a pool of worker processes that execute tasks concurrently.
+- `.map()`: Distributes the function (`cube_number`) across the process pool for all the numbers in the list.
+
+In this example, the `ProcessPoolExecutor` runs multiple processes concurrently, calculating the cube of each number in parallel.
+
+---
+
+## When to Use `multiprocessing` vs `concurrent.futures`?
+
+- Use `multiprocessing` when you need more control over each process (e.g., starting, stopping, or managing multiple processes manually).
+- Use `concurrent.futures.ProcessPoolExecutor` when you want a simpler, high-level interface for managing multiple CPU-bound tasks concurrently.
+
+---
+
+## Key Considerations:
+- **Inter-Process Communication (IPC)**: Since each process runs independently, sharing data between processes can be done via `multiprocessing.Queue` or `multiprocessing.Pipe`.
+- **Use Cases**: Multicore processing is ideal for CPU-bound tasks like numerical computations, data analysis, and heavy processing jobs that require multiple CPU cores.
+
+---
+
+## Performance Considerations:
+- **Overhead**: Starting new processes has more overhead than starting new threads, so multicore processing is best suited for long-running, CPU-bound tasks.
+- **Shared Memory**: By default, each process has its own memory space, which means you need to use specific techniques like `multiprocessing.Manager` for shared data.
+
+---
+
+### Further Reading:
+- [Python multiprocessing documentation](https://docs.python.org/3/library/multiprocessing.html)
+- [concurrent.futures documentation](https://docs.python.org/3/library/concurrent.futures.html)
+
+Note:- It is rephrased by ChatGPT 4o

2) Multicore Processing/basics/1)_multicore_programming_basics.py

@@ -0,0 +1,28 @@
+import multiprocessing
+import time
+import utils
+import os
+
+
+if __name__=="__main__":
+    img_file='img_src'
+    img_num=5
+    start_t=time.time()
+    utils.download_img_seq(img_file, img_num)
+    int_t=time.time()
+    print(f"Time taken for single core to download {img_num} images is {int_t-start_t} sec")
+
+    start_t=time.time()
+    processes=[]
+    for i in range(img_num):
+        img_file=utils.create_folder(img_file)
+        file_name=os.path.join(img_file,f"img-mc-{i+1}.jpg")
+
+        p=multiprocessing.Process(target=utils.download_img, args=[utils.url,file_name])
+        p.start()
+        processes.append(p)
+
+    for p in processes:
+        p.join()
+    end_t=time.time()
+    print(f"Time taken for multicore to download {img_num} images is {end_t-start_t} sec")