# GoPool

GoPool is a high-performance, feature-rich, and easy-to-use worker pool library for Golang. It is designed to manage and recycle a pool of goroutines to complete tasks concurrently, improving the efficiency and performance of your applications.

## Features

• Task Queue: GoPool uses a thread-safe task queue to store tasks waiting to be processed. Multiple workers can simultaneously fetch tasks from this queue.

• Dynamic Worker Adjustment: GoPool can dynamically adjust the number of workers based on the number of tasks and system load.

• Graceful Shutdown: GoPool can shut down gracefully. It stops accepting new tasks and waits for all ongoing tasks to complete before shutting down when there are no more tasks or a shutdown signal is received.

• Error Handling: GoPool can handle errors that occur during task execution. For example, it can provide an error callback function.

• Task Timeout Handling: GoPool can handle task execution timeouts. For example, it can set a timeout period, and if a task is not completed within this period, the task is considered failed.

• Task Priority: GoPool supports task priority. Tasks with higher priority are processed first.

• Task Result Retrieval: GoPool provides a way to retrieve task results. For example, it can provide a result callback function.

• Task Retry: GoPool provides a retry mechanism for failed tasks. For example, it can set the number of retries and the retry interval.

• Task Progress Tracking: GoPool provides task progress tracking. For example, it can provide a progress callback function or a method to query the current task progress.

• Concurrency Control: GoPool can control the number of concurrent tasks to prevent system overload.

## Installation

To install GoPool, use go get:

go get -u github.com/devchat-ai/gopool

## Usage

Here is a simple example of how to use GoPool:

// code example here

## Contributing

We welcome contributions! Please see CONTRIBUTING.md for details.

## License

GoPool is released under the MIT License. See LICENSE for details.

import ( "sync" "testing" "time"

"github.com/daniel-hutao/spinlock"

)

func TestGoPoolWithMutex(t *testing.T) { pool := NewGoPool(100, WithLock(new(sync.Mutex))) for i := 0; i < 1000; i++ { pool.AddTask(func() { time.Sleep(10 * time.Millisecond) }) } pool.Release() }

func TestGoPoolWithSpinLock(t *testing.T) { pool := NewGoPool(100, WithLock(new(spinlock.SpinLock))) for i := 0; i < 1000; i++ { pool.AddTask(func() { time.Sleep(10 * time.Millisecond) }) } pool.Release() }

func BenchmarkGoPoolWithMutex(b *testing.B) { var wg sync.WaitGroup var taskNum = int(1e6) pool := NewGoPool(5e4, WithLock(new(sync.Mutex)))

b.ResetTimer()
for i := 0; i < b.N; i++ {
    wg.Add(taskNum)
    for num := 0; num < taskNum; num++ {
        pool.AddTask(func() {
            time.Sleep(10 * time.Millisecond)
            wg.Done()
        })
    }
}
wg.Wait()
b.StopTimer()
pool.Release()

}

func BenchmarkGoPoolWithSpinLock(b *testing.B) { var wg sync.WaitGroup var taskNum = int(1e6) pool := NewGoPool(5e4, WithLock(new(spinlock.SpinLock)))

b.ResetTimer()
for i := 0; i < b.N; i++ {
    wg.Add(taskNum)
    for num := 0; num < taskNum; num++ {
        pool.AddTask(func() {
            time.Sleep(10 * time.Millisecond)
            wg.Done()
        })
    }
}
wg.Wait()
b.StopTimer()
pool.Release()

}

func BenchmarkGoroutines(b *testing.B) { var wg sync.WaitGroup var taskNum = int(1e6)

for i := 0; i < b.N; i++ {
    wg.Add(taskNum)
    for num := 0; num < taskNum; num++ {
        go func() {
            time.Sleep(10 * time.Millisecond)
            wg.Done()
        }()
    }
}

}

import ( "sync" )

type Task func()

type GoPool struct { Workers []*Worker MaxWorkers int workerStack []int taskQueue chan Task lock sync.Locker cond *sync.Cond }

type Option func(*GoPool)

func WithLock(lock sync.Locker) Option { return func(p *GoPool) { p.lock = lock p.cond = sync.NewCond(p.lock) } }

func NewGoPool(maxWorkers int, opts ...Option) *GoPool { pool := &GoPool{ MaxWorkers: maxWorkers, Workers: make([]*Worker, maxWorkers), workerStack: make([]int, maxWorkers), taskQueue: make(chan Task, 1e6), lock: new(sync.Mutex), } for _, opt := range opts { opt(pool) } if pool.cond == nil { pool.cond = sync.NewCond(pool.lock) } for i := 0; i < maxWorkers; i++ { worker := newWorker() pool.Workers[i] = worker pool.workerStack[i] = i worker.start(pool, i) } go pool.dispatch() return pool }

func (p *GoPool) AddTask(task Task) { p.taskQueue <- task }

func (p *GoPool) Release() { close(p.taskQueue) p.cond.L.Lock() for len(p.workerStack) != p.MaxWorkers { p.cond.Wait() } p.cond.L.Unlock() for _, worker := range p.Workers { close(worker.TaskQueue) } p.Workers = nil p.workerStack = nil }

func (p *GoPool) popWorker() int { p.lock.Lock() workerIndex := p.workerStack[len(p.workerStack)-1] p.workerStack = p.workerStack[:len(p.workerStack)-1] p.lock.Unlock() return workerIndex }

func (p *GoPool) pushWorker(workerIndex int) { p.lock.Lock() p.workerStack = append(p.workerStack, workerIndex) p.lock.Unlock() p.cond.Signal() }

func (p *GoPool) dispatch() { for task := range p.taskQueue { p.cond.L.Lock() for len(p.workerStack) == 0 { p.cond.Wait() } p.cond.L.Unlock() workerIndex := p.popWorker() p.Workers[workerIndex].TaskQueue <- task } }

type Worker struct { TaskQueue chan Task }

func newWorker() *Worker { return &Worker{ TaskQueue: make(chan Task, 1), } }

func (w *Worker) start(pool *GoPool, workerIndex int) { go func() { for task := range w.TaskQueue { if task != nil { task() } pool.pushWorker(workerIndex) } }() }

GoPool is a high-performance, feature-rich, and easy-to-use worker pool library for Golang. It is designed to manage and recycle a pool of goroutines to complete tasks concurrently, improving the efficiency and performance of your applications.

Features

• Task Queue: GoPool uses a thread-safe task queue to store tasks waiting to be processed. Multiple workers can simultaneously fetch tasks from this queue.

• Dynamic Worker Adjustment: GoPool can dynamically adjust the number of workers based on the number of tasks and system load.

• Graceful Shutdown: GoPool can shut down gracefully. It stops accepting new tasks and waits for all ongoing tasks to complete before shutting down when there are no more tasks or a shutdown signal is received.

• Error Handling: GoPool can handle errors that occur during task execution. For example, it can provide an error callback function.

• Task Timeout Handling: GoPool can handle task execution timeouts. For example, it can set a timeout period, and if a task is not completed within this period, the task is considered failed.

• Task Priority: GoPool supports task priority. Tasks with higher priority are processed first.

• Task Result Retrieval: GoPool provides a way to retrieve task results. For example, it can provide a result callback function.

• Task Retry: GoPool provides a retry mechanism for failed tasks. For example, it can set the number of retries and the retry interval.

• Task Progress Tracking: GoPool provides task progress tracking. For example, it can provide a progress callback function or a method to query the current task progress.

• Concurrency Control: GoPool can control the number of concurrent tasks to prevent system overload.

• Lock Customization: GoPool supports different types of locks. You can use the built-in sync.Mutex or a custom lock such as spinlock.SpinLock.

Installation

To install GoPool, use go get:

go get -u github.com/devchat-ai/gopool

Usage

Here is a simple example of how to use GoPool:

package main

import (
    "sync"
    "time"

    "github.com/devchat-ai/gopool"
)

func main() {
    pool := gopool.NewGoPool(100, gopool.WithLock(new(sync.Mutex)))
    for i := 0; i < 1000; i++ {
        pool.AddTask(func() {
            time.Sleep(10 * time.Millisecond)
        })
    }
    pool.Release()
}