Skip to content

v0.6.3 go.sdk 是否不支持,v0.6.2正常 #1556

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
hyx52077 opened this issue Mar 20, 2025 · 0 comments
Open

v0.6.3 go.sdk 是否不支持,v0.6.2正常 #1556

hyx52077 opened this issue Mar 20, 2025 · 0 comments
Labels
bug Something isn't working

Comments

@hyx52077
Copy link

hyx52077 commented Mar 20, 2025

Bug 描述

go.mod

go 1.23.7

require (
	github.com/apex/log v1.8.0
	github.com/crawlab-team/crawlab-go-sdk v0.0.6
	github.com/go-redis/redis/v8 v8.11.5
	github.com/gocolly/colly/v2 v2.1.0
)

以下对v0.6.3 go sdk v0.0.6 文件 执行存在的bug

~/go/pkg/mod/github.com/crawlab-team/[email protected]/utils/data.go

  • task_id 字段得改之外
func SaveItemMongo(item entity.Item) (err error) {
	ds := GetDataSource()
	_, c, err := database.GetMongoCol(ds)
	item["task_id"] = GetTaskId()

~/go/pkg/mod/github.com/crawlab-team/[email protected]/database/mongo.go

  • os.Getenv("CRAWLAB_COLLECTION")
  • 环境变量 CRAWLAB_COLLECTION 执行任务时 需要一个.sh文件 手动赋值环境变量 感觉在应该v0.6.3 环境变量名变了,v0.6.2 正常获取
func GetMongoCol(ds entity.DataSource) (*mgo.Session, *mgo.Collection, error) {
	if ds.Type == "" {
		return GetDataSourceCol(
			os.Getenv("CRAWLAB_MONGO_HOST"),
			os.Getenv("CRAWLAB_MONGO_PORT"),
			os.Getenv("CRAWLAB_MONGO_USERNAME"),
			os.Getenv("CRAWLAB_MONGO_PASSWORD"),
			os.Getenv("CRAWLAB_MONGO_AUTHSOURCE"),
			os.Getenv("CRAWLAB_MONGO_DB"),
			os.Getenv("CRAWLAB_COLLECTION"),
		)
	}
	return GetDataSourceCol(
		ds.Host,
		ds.Port,
		ds.Username,
		ds.Password,
		ds.AuthSource,
		ds.Database,
		os.Getenv("CRAWLAB_COLLECTION"),
	)
}

v0.6.3 镜像 crawlabteam/crawlab:latest

    • 上传go编译二进制文件 日志显示没有执行权限,
      手动进入容器更改了权限可以执行,可以入库,但貌似入库时设置的字段环境变量taskid字段值不对。是否sdk暂不支持v0.6.3
    • 上传zip不会自动解压

v0.6.2 镜像 tikazyq/crawlab:latest

  • 以上都正常

我的 .sh 脚本文件

GOOS=linux GOARCH=amd64 go build -o main main.go
chmod 755 ./main
# 压缩 zip ./main ./config.yaml
zip -r main.zip ./main ./config.yml

复现步骤
该 Bug 复现步骤如下

  1. 任务命令 ./main

期望结果
xxx 能工作。

@hyx52077 hyx52077 added the bug Something isn't working label Mar 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant