Performance and scalability improvements

[danielinux]

• more tests
• replaced poll() with epoll() in libevquick
• replaced malloc with a memory pool to reduce allocations under pressure
• client lookup using hash table O(n) -> O(1)
• update to TLS 1.3 + faster resume handshake via session tickets
• Add 5s timeout for pending requests to get rid of orphan requests / fix requests leaks

[Copilot]

Pull request overview

This PR implements significant performance and scalability improvements to the DNS-over-HTTPS daemon (dohd), focusing on reducing memory allocation overhead, improving client lookup performance, and modernizing the TLS stack. The changes address scalability bottlenecks and add comprehensive testing infrastructure for leak detection and stress testing.

Changes:

• Replaced poll() with epoll() in libevquick for better scalability under high connection loads
• Introduced memory pools for client_data and req_slot to reduce malloc/free overhead (max 1024 clients, 4096 requests)
• Replaced O(n) client list traversal with O(1) hash table lookups using a 1024-bucket hash table
• Upgraded from TLS 1.2 to TLS 1.3 with session ticket support for faster reconnection via 1-RTT resume
• Added 5-second timeout for DNS requests to prevent orphaned requests and fix request leaks
• Added comprehensive test suite including unit tests, integration tests, stress tests, and memory leak detection tools (valgrind, ASAN)

Reviewed changes

Copilot reviewed 22 out of 23 changed files in this pull request and generated 17 comments.

Show a summary per file

File	Description
src/mempool.h, src/mempool.c	New fixed-size memory pool implementation with freelist-based O(1) allocation/deallocation
src/libevquick.c	Migrated from poll() to epoll() event multiplexing with deferred event cleanup to prevent use-after-free
src/dohd.c	Added client hash table, memory pool integration, request timeouts, TLS 1.3 upgrade, and session tickets
src/Makefile	Updated to build with new mempool.o object file
test/test_mempool.c	Comprehensive unit tests for memory pool (487 lines, 22 test cases)
test/test_heap.c	Unit tests for min-heap timer implementation (286 lines, 11 test cases)
test/test_dns_parser.c	Unit tests for DNS packet parsing and TTL extraction (375 lines, 8 test cases)
test/test_url64_extended.c	Extended unit tests for URL-safe base64 decoding (225 lines, 10 test cases)
test/valgrind_test.sh	Automated valgrind leak detection test with test sequence
test/run_asan_test.sh	AddressSanitizer integration test wrapper
test/gen_test_certs.sh	Automated self-signed certificate generation for testing
test/test_client_lifecycle.sh	Integration test for rapid client connect/disconnect cycles
test/test_request_leaks.sh	Memory leak detection via stats monitoring and high request volume
test/test_concurrent.sh	Concurrent connection stress test measuring requests per second
test/stress_test.sh	Multi-threaded stress test running until failure detection
test/stress_escalate.sh	Escalating stress test to find maximum sustainable load
test/stress_flood.sh	Connection flood test for resource exhaustion scenarios
test/stress_chaos.sh	Chaos test with random connection patterns to find edge cases
test/Makefile	Build configuration for all unit and integration tests
test/README.md	Comprehensive testing documentation (193 lines)
Makefile	Added targets for various test suites (check, check-asan, stress-*, etc.)
.gitignore	Added test binaries and generated certificates to ignore list

Comments suppressed due to low confidence (1)

src/libevquick.c:378

• Multiple resource leaks in the evquick_init function error paths. When initialization fails after allocating resources, the function returns NULL without cleaning up previously allocated resources:

1. If pipe() succeeds but epoll_create1() fails, the pipe file descriptors leak (lines 346, 352-355)
2. If epoll_create1() succeeds but epoll_ctl() fails, the epfd leaks (lines 352, 361-364) - though close(ctx->epfd) is called, the ctx itself and timers are not freed
3. If sigaction() fails, ctx, timers, pipe, and epfd all leak (lines 371-373)

Add proper cleanup in error paths: free heap_timers, close pipe fds, close epfd, free ctx.

CTX evquick_init(void)
{
    int yes = 1;
    struct sigaction act;
    struct epoll_event ev;
#ifdef EVQUICK_PTHREAD
    CTX ctx;
    pthread_mutex_init(&ctx_list_mutex, NULL);
#endif
    ctx = calloc(1, sizeof(struct evquick_ctx));
    if (!ctx)
        return NULL;
    ctx->giveup = 0;
    ctx->timers = heap_init();
    if (!ctx->timers)
        return NULL;
    if(pipe(ctx->time_machine) < 0)
        return NULL;
    (void)yes;
    fcntl(ctx->time_machine[1], F_SETFL, O_NONBLOCK);

    /* Create epoll instance */
    ctx->epfd = epoll_create1(0);
    if (ctx->epfd < 0) {
        perror("epoll_create1");
        return NULL;
    }

    /* Add time_machine pipe to epoll for timer wakeups */
    ev.events = EPOLLIN;
    ev.data.ptr = NULL;  /* NULL ptr indicates time_machine */
    if (epoll_ctl(ctx->epfd, EPOLL_CTL_ADD, ctx->time_machine[0], &ev) < 0) {
        perror("epoll_ctl time_machine");
        close(ctx->epfd);
        return NULL;
    }

    ctx->n_events = 1;
    memset(&act, 0, sizeof(act));
    act.sa_handler = sig_alrm_handler;
    act.sa_flags = SA_NODEFER;
    if (sigaction(SIGALRM, &act, NULL) < 0) {
        perror("Setting alarm signal");
        return NULL;
    }
    ctx_add(ctx);
    timer_new(ctx);
    return ctx;
}

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.