home/crawlab
2
0
Fork 0
mirror of https://github.com/wahyd4/crawlab.git synced 2026-08-27 14:06:18 +10:00
Code Issues Packages Projects Releases Wiki Activity
792 Commits 5 Branches 29 Tags
169a7d0df90ce109705c558d0feabbebdae73fcd
Commit Graph
792 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
陈景阳 169a7d0df9 fix 可能直接退出的问题 2019-09-26 16:49:01 +08:00
陈景阳 31be4c1839 优化爬虫获取逻辑 2019-09-26 16:43:32 +08:00
陈景阳 a11544d809 优化爬虫获取逻辑 2019-09-26 16:26:32 +08:00
陈景阳 947b561653 优化爬虫获取逻辑 2019-09-26 11:38:13 +08:00
陈景阳 05c28230b7 爬虫逻辑修改为从GridFS获取 2019-09-26 11:28:20 +08:00
陈景阳 96e926b486 fix 无法终止任务的问题 2019-09-26 10:29:58 +08:00
陈景阳 e235076a11 fix 无法终止任务的问题 2019-09-26 10:23:08 +08:00
陈景阳 7218a4f4de 添加日志打印 2019-09-26 09:37:10 +08:00
陈景阳 fda7e56e1a 添加日志打印 2019-09-25 16:19:58 +08:00
陈景阳 82999225de fix 无法获取task的错误 2019-09-25 15:11:27 +08:00
陈景阳 951f6a9f07 fix 2019-09-25 14:38:46 +08:00
暗音andGitHub 0cb31ad96e Merge pull request #3 from crawlab-team/develop 
Develop
 2019-09-25 09:27:15 +08:00
Marvin ZhangandGitHub a4b69f69a6 Merge pull request #224 from wo10378931/develop 
Develop
 2019-09-10 16:24:10 +08:00
陈景阳 3a272b404c fix 空指针引用 2019-09-10 14:47:23 +08:00
陈景阳 42e7647cd4 fix 消息无法订阅问题 
fix 可能出现重复爬虫的问题
 2019-09-10 14:26:50 +08:00
Marvin ZhangandGitHub b45b7df6e4 Merge pull request #223 from yaziming/fix-redis-subscrible-failed-problem 
fix(backend):  redis subscrible channel name is wrong.
 2019-09-10 13:08:05 +08:00
yaziming 3976dc568f fix(all): fix redis subscrible wrong channel 
fix redis subscrible wrong channel

Closes #222
 2019-09-10 12:55:14 +08:00
陈景阳 000b4ff730 删除日志 2019-09-10 09:51:27 +08:00
陈景阳 cdeef4b4a5 重构msg的包 2019-09-10 09:32:48 +08:00
陈景阳 8cf48d871e fix 删除spider没有删除task 2019-09-05 20:07:08 +08:00
Marvin ZhangandGitHub 2e496964fd Merge pull request #214 from crawlab-team/master 
Master to Develop
 2019-09-04 20:57:15 +08:00
Marvin ZhangandGitHub 166c49550c Merge pull request #213 from yaziming/forked_master 
Merge  master
 2019-09-04 20:56:16 +08:00
yaziming 2bd8891889 Merge remote-tracking branch 'forked/develop' into forked_master 2019-09-04 20:42:01 +08:00
yaziming 5e2b3ca228 Revert "Revert "V0.4.0 imporve error response"" 
This reverts commit acccdb65bd.
 2019-09-04 20:41:37 +08:00
Marvin Zhang 5013233481 added param to task 2019-09-04 20:13:28 +08:00
Marvin Zhang 4893439ffb added param to task 2019-09-04 20:13:12 +08:00
Marvin ZhangandGitHub 1b7eea7e57 Merge pull request #210 from hantmac/develop 
bug fix:fix data race caused by LockList
 2019-09-04 14:14:45 +08:00
hantmac 2fb1754e56 bug fix:fix data race caused by LockList 2019-09-04 13:51:02 +08:00
Marvin ZhangandGitHub 4e17158a46 Merge pull request #208 from yaziming/v0.4.0-imporve-error-response 
feat(backend): add some feat for error message system
 2019-09-03 15:25:04 +08:00
Marvin ZhangandGitHub a5449a674a Merge pull request #209 from crawlab-team/revert-207-v0.4.0-imporve-error-response 
Revert "V0.4.0 imporve error response"
 2019-09-03 15:24:40 +08:00
Marvin ZhangandGitHub acccdb65bd Revert "V0.4.0 imporve error response" 2019-09-03 15:24:26 +08:00
Marvin ZhangandGitHub e591986275 Merge pull request #207 from yaziming/v0.4.0-imporve-error-response 
V0.4.0 imporve error response
 2019-09-03 15:24:06 +08:00
Marvin ZhangandGitHub 53cce33d64 Merge pull request #206 from yaziming/v0.4.0-use-unsafe.pointer-convert-bytes-to-string 
refactor(backend): Use more efficient bytes to string methods and remove unnecessary type conversions
 2019-09-03 15:23:37 +08:00
yaziming 71c83251e5 feat(backend): add some feat for error message system 
errors support custom http code and support return invalid validated
error
 2019-09-03 15:23:08 +08:00
yaziming e10d8fd996 refactor(backend): Use more efficient bytes to string methods and remove unnecessary type conversions 
detail:
    1. add utils.BytesToString function instead of string() convert bytes to string.
    2. use bytes.NewReader instead of strings.NewReader(string(sb)).
    3. use w.Body.Bytes() instead of []byte(w.Body.String()).
 2019-09-03 15:17:32 +08:00
Marvin ZhangandGitHub ae9f17716f Merge pull request #205 from hantmac/develop 
bug fix:fix go test ./..
 2019-09-03 13:41:00 +08:00
hantmac cb3d7263b2 bug fix:fix go test ./.. bug 2019-09-03 12:47:18 +08:00
Marvin ZhangandGitHub 3c244c7306 Merge pull request #204 from wo10378931/develop 
fix 几个问题
 2019-09-03 09:11:46 +08:00
陈景阳 cd78e6c745 还原代码 2019-09-03 09:06:04 +08:00
陈景阳 eb92d34f9b Merge branch 'develop' of https://github.com/crawlab-team/crawlab into develop 2019-09-03 08:57:21 +08:00
Marvin ZhangandGitHub ce08411960 Merge pull request #203 from yaziming/v0.4.0-mongo-redis-support-timeout-and-pool 
add timeout and pool
 2019-09-03 08:19:22 +08:00
yaziming 361134f8dd Remove redundant fmt.Println 2019-09-02 22:57:29 +08:00
陈景阳 352fac4096 fix 删除spider没有删除task的问题 2019-09-02 20:49:00 +08:00
yaziming 10c3488caf break loop when subscribe count is 0 2019-09-02 18:32:47 +08:00
陈景阳 b5084c964b 还原代码 2019-09-02 18:14:34 +08:00
陈景阳 8b237ddc2a fix 无法正常删除有问题的爬虫 2019-09-02 18:04:47 +08:00
陈景阳 93dd3b714a fix 如果从dir读取爬虫为空，则移除所有的爬虫 2019-09-02 17:37:48 +08:00
陈景阳 f9fdf5b8d8 fix 打印日志只有2KB 2019-09-02 16:45:56 +08:00
陈景阳 033ca493aa 添加排序 2019-09-02 10:40:16 +08:00
陈景阳 eaf412dc80 Merge branch 'develop' of https://github.com/crawlab-team/crawlab into develop 2019-09-02 09:55:49 +08:00