我在这里遇到了一些麻烦,以了解线程是如何工作的或者是如何构build的。
我需要下载几个网页,更改链接(容易部分)的一个值,并获得一些信息,但我使用“时”,它需要大约1秒或更多的下载一个像60kb的东西的网站…我的互联网5MB ..
有人能给我提供最简单的例子吗?
来自高级使用:异步请求
from requests import async urls = [ 'http://python-requests.org', 'http://httpbin.org', 'http://python-guide.org', 'http://kennethreitz.com' ] rs = [async.get(u) for u in urls] async.map(rs)
这不是使用线程,但工作原理是相同的 – 请求正在同时进行。
从这里提取
#!/usr/bin/env python import Queue import threading import urllib2 import time hosts = ["http://yahoo.com", "http://google.com", "http://amazon.com", "http://ibm.com", "http://apple.com"] queue = Queue.Queue() class ThreadUrl(threading.Thread): """Threaded Url Grab""" def __init__(self, queue): threading.Thread.__init__(self) self.queue = queue def run(self): while True: #grabs host from queue host = self.queue.get() #grabs urls of hosts and prints first 1024 bytes of page url = urllib2.urlopen(host) print url.read(1024) #signals to queue job is done self.queue.task_done() start = time.time() def main(): #spawn a pool of threads, and pass them queue instance for i in range(5): t = ThreadUrl(queue) t.setDaemon(True) t.start() #populate queue with data for host in hosts: queue.put(host) #wait on the queue until everything has been processed queue.join() main() print "Elapsed Time: %s" % (time.time() - start)