python使用rabbitmq实现网络爬虫示例

编写tasks.py复制代码 代码如下:from celery import Celeryfrom tornado.httpclient import HTTPClientapp = Celery(‘tasks’)app.config_from_object(‘celeryconfig’)@app.taskdef get_html(url): http_client = HTTPClient() try: response = http_client.fetch(url,follow_redirects=True) return response.bod