python joblib在Windows平行并不工作甚至“if __name__ =='__main__':”被添加

我在Windows上运行Python中的并行处理。 这是我的代码:

from joblib import Parallel, delayed def f(x): return sqrt(x) if __name__ == '__main__': a = Parallel(n_jobs=2)(delayed(f)(i) for i in range(10)) 

这里是错误信息:

 Process PoolWorker-2: Process PoolWorker-1: Traceback (most recent call last): File "C:\Users\yoyo__000.BIGBLACK\AppData\Local\Enthought\Canopy\App\appdata\canopy-1.5.4.3105.win-x86_64\lib\multiprocessing\process.py", line 258, in _bootstrap self.run() File "C:\Users\yoyo__000.BIGBLACK\AppData\Local\Enthought\Canopy\App\appdata\canopy-1.5.4.3105.win-x86_64\lib\multiprocessing\process.py", line 114, in run self._target(*self._args, **self._kwargs) File "C:\Users\yoyo__000.BIGBLACK\AppData\Local\Enthought\Canopy\App\appdata\canopy-1.5.4.3105.win-x86_64\lib\multiprocessing\pool.py", line 102, in worker task = get() File "C:\Users\yoyo__000.BIGBLACK\AppData\Local\Enthought\Canopy\User\lib\site-packages\joblib\pool.py", line 363, in get return recv() AttributeError: 'module' object has no attribute 'f' 

根据这个网站的问题是Windows特定的:

是的:在linux下我们是分叉的,因此他们不需要腌制函数,而且工作正常。 在windows下,该功能需要被pickleable,即它需要从另一个文件导入。 这实际上是一个很好的做法:使模块推动重用。

我试过你的代码,它在Linux下完美地工作。 在Windows下,如果它是从脚本运行的,就像python script_with_your_code.py 。 但是在交互式Python会话中运行失败。 当我将f函数保存在单独的模块中并将其导入到交互式会话中时,它对我有用。

不工作:

互动会议:

 >>> from math import sqrt >>> from joblib import Parallel, delayed >>> def f(x): ... return sqrt(x) >>> if __name__ == '__main__': ... a = Parallel(n_jobs=2)(delayed(f)(i) for i in range(10)) ... Process PoolWorker-1: Traceback (most recent call last): File "C:\Python27\lib\multiprocessing\process.py", line 258, in _bootstrap self.run() File "C:\Python27\lib\multiprocessing\process.py", line 114, in run self._target(*self._args, **self._kwargs) File "C:\Python27\lib\multiprocessing\pool.py", line 102, in worker task = get() File "C:\Python27\lib\site-packages\joblib\pool.py", line 359, in get return recv() AttributeError: 'module' object has no attribute 'f' 

加工:
fun.py

 from math import sqrt def f(x): return sqrt(x) 

互动会议:

 >>> from joblib import Parallel, delayed >>> from fun import f >>> if __name__ == '__main__': ... a = Parallel(n_jobs=2)(delayed(f)(i) for i in range(10)) ... >>> a [0.0, 1.0, 1.4142135623730951, 1.7320508075688772, 2.0, 2.23606797749979, 2.449489742783178, 2.6457513110645907, 2.8284271247461903, 3.0]