TypechoJoeTheme

运维茶馆

统计
登录
用户名
密码

centos7安装pyspider

少延博主
2022-01-07
/
0 评论
/
165 阅读
/
49 个字
/
百度已收录
01/07
本文最后更新于2022年01月26日,已超过155天没有更新。如果文章内容或图片资源失效,请留言反馈,我会及时处理,谢谢!

安装

## 安装
[root@ac4:~]$ pip3 install pyspider
##  运行,报错
[root@ac4:~]$ pyspider
ValueError: Invalid configuration:
  - Deprecated option 'domaincontroller': use 'http_authenticator.domain_controller' instead.

## 解决:
[root@ac4:~]$ pip3 install wsgidav==2.4.1
##  再次运行,报错
[root@ac4:~]$ pyspider
    from werkzeug.wsgi import DispatcherMiddleware
ImportError: cannot import name 'DispatcherMiddleware'
## 解决。先降低 flask 版本,再降低 werkzeug 的版本
[root@ac4:~]$  pip3 install flask==1.0
[root@ac4:~]$  pip3 install werkzeug==0.16.1

## 再次运行,成功。
[root@ac4:~]$ pyspider
[W 220107 11:35:30 run:413] phantomjs not found, continue running without it.
[I 220107 11:35:32 result_worker:49] result_worker starting...
[I 220107 11:35:33 tornado_fetcher:638] fetcher starting...
[I 220107 11:35:33 processor:211] processor starting...
[I 220107 11:35:33 scheduler:647] scheduler starting...
[I 220107 11:35:33 scheduler:782] scheduler.xmlrpc listening on 127.0.0.1:23333
[I 220107 11:35:33 scheduler:586] in 5m: new:0,success:0,retry:0,failed:0
[I 220107 11:35:33 app:76] webui running on 0.0.0.0:5000

配置访问密码

## 准备配置文件
[root@ac4:/data/app/pyspider]$ cat config.json 
{
    "webui": {
        "port": "5000",
        "username": "abc",
        "password": "123456",
        "need-auth": true
    }
}
## 运行
[root@ac4:/data/app/pyspider]$ pyspider --config config.json 
[W 220107 11:42:07 run:413] phantomjs not found, continue running without it.
[I 220107 11:42:09 result_worker:49] result_worker starting...
[I 220107 11:42:10 tornado_fetcher:638] fetcher starting...
[I 220107 11:42:10 processor:211] processor starting...
[I 220107 11:42:10 scheduler:647] scheduler starting...
[I 220107 11:42:10 scheduler:782] scheduler.xmlrpc listening on 127.0.0.1:23333
[I 220107 11:42:10 scheduler:586] in 5m: new:0,success:0,retry:0,failed:0
[I 220107 11:42:10 app:76] webui running on 0.0.0.0:5000



赞(0)
赞赏
感谢您的支持,我会继续努力哒!
版权属于:

运维茶馆

本文链接:

https://www.opstea.com/archives/pyspider.html(转载时请注明本文出处及文章链接)

评论 (0)
本篇文章评论功能已关闭