Threading vs Multiprocessing in Jupyter
Medir el tiempo de ejecución usando Threads y Multiprocess
El uso de la programación concurrente en Python es interesante y sorprendente. Experimentando con Carnets y Google Colaboratory he llegado a la conclusión que su uso se justifica según el problema a resolver.
En Carnet el uso de multiprocessing no esta soportado hasta ahora. Su creador Nicolas Holzschuch comenta lo siguiente:
The short answer is:"you can't use multiprocessing with Carnets". I'm sorry about that, but it is really too difficult to support.> >The long answer is:multiprocessing in Python relies on multiple calls to fork(), with a pool of process and a queue for tasks to be executed. iOS does not have the fork() system call, so there is no way to make this work, and I don't think there is one.> Since most iPads have only 4 cores anyway (and some cores are already used by the system and the Jupyter server), I am not certain we would gain much processing power anyway.
En las siguientes celdas puedes observar y experimentar como se comporta el tiempo de ejecución en Jupyter usando Threading y Multiprocessing
%%writefile fibonacci.py
def fibonacci(n):
if n < 2:
return n
return fibonacci(n - 1) + fibonacci(n - 2)
from concurrent.futures import ThreadPoolExecutor
from fibonacci import fibonacci
def fibs(n):
"""Fibonacci sequentially"""
for i in range(1, n + 1):
_ = fibonacci(i)
def fibs_thread(n):
"""Fibonacci with Threads"""
with ThreadPoolExecutor(max_workers=4) as exct:
exct.map(fibonacci, range(1, n + 1))
if __name__ == "__main__":
N=30
%timeit fibs(N)
print()
%timeit fibs_thread(N)
from multiprocessing import Pool, cpu_count
from fibonacci import fibonacci
def fibs_multiprocess(n):
"""Fibonacci with CPU Multiprocess"""
with Pool(processes=cpu_count()) as cpu_pool:
cpu_pool.map(fibonacci, range(1, n + 1))
if __name__ == "__main__":
N=30
%timeit fibs_multiprocess(N)