Python多处理只使用一个核心

我正在尝试从标准的python文档中的代码片段来学习如何使用多处理模块。 该代码粘贴在此消息的末尾。 我在四核机器上的Ubuntu 11.04上使用Python 2.7.1(根据系统监视器给我八个由于超线程引起的内核)

问题:尽pipe有几个进程已经启动,但是所有的工作量似乎都只被安排到一个核心,即接近100%的利用率。 偶尔,所有工作负载都会迁移到另一个核心,但工作负载绝不会分配到其中。

任何想法,为什么这样呢?

最好的祝福,

保罗

# # Simple example which uses a pool of workers to carry out some tasks. # # Notice that the results will probably not come out of the output # queue in the same in the same order as the corresponding tasks were # put on the input queue. If it is important to get the results back # in the original order then consider using `Pool.map()` or # `Pool.imap()` (which will save on the amount of code needed anyway). # # Copyright (c) 2006-2008, R Oudkerk # All rights reserved. # import time import random from multiprocessing import Process, Queue, current_process, freeze_support # # Function run by worker processes # def worker(input, output): for func, args in iter(input.get, 'STOP'): result = calculate(func, args) output.put(result) # # Function used to calculate result # def calculate(func, args): result = func(*args) return '%s says that %s%s = %s' % \ (current_process().name, func.__name__, args, result) # # Functions referenced by tasks # def mul(a, b): time.sleep(0.5*random.random()) return a * b def plus(a, b): time.sleep(0.5*random.random()) return a + b def test(): NUMBER_OF_PROCESSES = 4 TASKS1 = [(mul, (i, 7)) for i in range(500)] TASKS2 = [(plus, (i, 8)) for i in range(250)] # Create queues task_queue = Queue() done_queue = Queue() # Submit tasks for task in TASKS1: task_queue.put(task) # Start worker processes for i in range(NUMBER_OF_PROCESSES): Process(target=worker, args=(task_queue, done_queue)).start() # Get and print results print 'Unordered results:' for i in range(len(TASKS1)): print '\t', done_queue.get() # Add more tasks using `put()` for task in TASKS2: task_queue.put(task) # Get and print some more results for i in range(len(TASKS2)): print '\t', done_queue.get() # Tell child processes to stop for i in range(NUMBER_OF_PROCESSES): task_queue.put('STOP') test() 

一些如何改变CPU亲和力。 之前我曾经遇到过这个问题。 我在这里找到了解决方案http://bugs.python.org/issue17038#msg180663

多处理并不意味着你会使用一个处理器的所有内核,你只是得到多个进程而不是多核进程,这将由操作系统来处理,是不确定的,@Devraj发表评论有问题的答案,以完成你欲望。

我找到了一个使用并行Python的工作。 我知道这不是使用基本Python库的解决方案,但代码很简单,就像一个魅力

尝试用真正需要CPU的东西替换time.sleep ,你会看到multiprocess工作正常! 例如:

 def mul(a, b): for i in xrange(100000): j = i**2 return a * b def plus(a, b): for i in xrange(100000): j = i**2 return a + b