Skip to content

perf: batch insert attemptDetail #122

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

vm-001
Copy link
Member

@vm-001 vm-001 commented Mar 14, 2025

Summary

This PR optimizes the attempt_details insertion performance.

  • using text type to get a slight performance improvement
  • using channel as queue to implement batch insertion

@vm-001 vm-001 requested a review from cchenggit March 14, 2025 08:59
Copy link

codecov bot commented Mar 14, 2025

Codecov Report

Attention: Patch coverage is 95.50562% with 4 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
db/entities/types.go 42.85% 4 Missing ⚠️
Flag Coverage Δ
integration 72.78% <95.50%> (+0.05%) ⬆️
unit 13.05% <0.00%> (-0.08%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
app/app.go 75.46% <100.00%> (ø)
db/dao/attempt_detail_dao.go 100.00% <100.00%> (ø)
dispatcher/dispatcher.go 73.94% <100.00%> (ø)
pkg/taskqueue/queue.go 66.66% <ø> (ø)
worker/worker.go 76.90% <100.00%> (+1.05%) ⬆️
db/entities/types.go 42.85% <42.85%> (-7.15%) ⬇️
🚀 New features to boost your workflow:
  • Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@vm-001 vm-001 force-pushed the perf/batch-insert-attempt-detail branch 11 times, most recently from 7fb42c9 to 31940e1 Compare March 14, 2025 10:41
@webhookx-x webhookx-x force-pushed the perf/batch-insert-attempt-detail branch from 31940e1 to 260b074 Compare March 18, 2025 10:20
@@ -446,3 +440,40 @@ func listEndpointPlugins(ctx context.Context, db *db.DB, endpointId string) ([]*
}
return *plugins, err
}

func (w *Worker) consumeAttemptDetails() {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpicking: consider graceful shutdown when context done.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants