Skip to content

🐛 bug: remove unbounded background goroutines from logger, cache, and internal storage memory#4378

Draft
Copilot wants to merge 6 commits into
mainfrom
copilot/fix-goroutine-leaks
Draft

🐛 bug: remove unbounded background goroutines from logger, cache, and internal storage memory#4378
Copilot wants to merge 6 commits into
mainfrom
copilot/fix-goroutine-leaks

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented May 28, 2026

Description

This change removes per-instance background goroutine patterns from logger and cache, while preserving the background GC in internal/storage/memory and making it cleanly stoppable through the storage’s existing lifecycle.

  • Logger

    • Replaced the ${time} background updater with lazy timestamp refresh during request handling.
    • Preserves the configured TimeFormat, timezone, and update interval semantics without spawning a goroutine per middleware instance.
  • Cache

    • Removed the cache timestamp ticker used only to maintain Age/expiration calculations.
    • Switched those code paths to read the current timestamp when processing requests.
  • Internal storage memory

    • Kept the background GC loop in internal/storage/memory.
    • Made shutdown safe with a buffered done channel and an idempotent Close() so storage instances can stop their GC goroutine without blocking or double-close issues.
  • Tests

    • Added focused coverage for logger timestamp refresh after the configured interval.
    • Added coverage that repeated internal/storage/memory.Close() calls do not block.
// cache/logger now use request-time timestamps instead of per-instance background updaters
ts := safeUnixSeconds(time.Now())

Changes introduced

List the new features or adjustments introduced in this pull request. Provide details on benchmarks, documentation updates, changelog entries, and if applicable, the migration guide.

  • Benchmarks: Describe any performance benchmarks and improvements related to the changes.
  • Documentation Update: Detail the updates made to the documentation and links to the changed files.
  • Changelog/What's New: Remove unbounded per-instance background goroutines from logger and cache, and make internal/storage/memory GC shutdown safe with a buffered stop signal and idempotent Close().
  • Migration Guide: If necessary, provide a guide or steps for users to migrate their existing code to accommodate these changes.
  • API Alignment with Express: Explain how the changes align with the Express API.
  • API Longevity: The change is internal-only and avoids introducing new public lifecycle APIs while fixing leaked goroutine and shutdown behavior within existing middleware/storage lifecycles.
  • Examples: Logger ${time} and cache expiration logic now refresh from request-time reads, while internal/storage/memory continues to reclaim expired entries in the background and now stops cleanly via Close().

Type of change

Please delete options that are not relevant.

  • New feature (non-breaking change which adds functionality)
  • Enhancement (improvement to existing features and functionality)
  • Documentation update (changes to documentation)
  • Performance improvement (non-breaking change which improves efficiency)
  • Code consistency (non-breaking change which improves code reliability and robustness)

Checklist

Before you submit your pull request, please make sure you meet these requirements:

  • Followed the inspiration of the Express.js framework for new functionalities, making them similar in usage.
  • Conducted a self-review of the code and provided comments for complex or critical parts.
  • Updated the documentation in the /docs/ directory for Fiber's documentation.
  • Added or updated unit tests to validate the effectiveness of the changes or new features.
  • Ensured that new and existing unit tests pass locally with the changes.
  • Verified that any new dependencies are essential and have been agreed upon by the maintainers/community.
  • Aimed for optimal performance with minimal allocations in the new code.
  • Provided benchmarks for the new code to analyze and improve upon.

Commit formatting

Please use emojis in commit messages for an easy way to identify the purpose or intention of a commit. Check out the emoji cheatsheet here: CONTRIBUTING.md

Copilot AI changed the title [WIP] Fix goroutine leaks in background processes 🐛 bug: remove unbounded background goroutines from logger, cache, and internal memory May 28, 2026
Copilot AI requested a review from ReneWerner87 May 28, 2026 20:36
@ReneWerner87 ReneWerner87 added this to v3 May 29, 2026
@ReneWerner87 ReneWerner87 added this to the v3 milestone May 29, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented May 29, 2026

Codecov Report

❌ Patch coverage is 90.00000% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.37%. Comparing base (f49e7d8) to head (8cef4d3).
⚠️ Report is 9 commits behind head on main.

Files with missing lines Patch % Lines
middleware/logger/logger.go 88.23% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4378      +/-   ##
==========================================
+ Coverage   91.33%   91.37%   +0.03%     
==========================================
  Files         132      132              
  Lines       13105    13112       +7     
==========================================
+ Hits        11970    11981      +11     
+ Misses        716      712       -4     
  Partials      419      419              
Flag Coverage Δ
unittests 91.37% <90.00%> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ReneWerner87
Copy link
Copy Markdown
Member

@copilot Thanks, this does remove the goroutine leak, but please reconsider the mechanism per case instead of removing all three goroutines uniformly.

Logger + cache timestamp: the original bug was N goroutines for N middleware instances. The cleanest fix is a single process-wide coarse clock: one background updater started once via sync.Once, shared by all instances, refreshing an atomically stored timestamp about once per second. Per-request cost stays a single atomic load (cheaper than calling time.Now() on every request), and there is no per-instance goroutine to leak. A fully lazy time.Now()-per-request also works, but the singleton coarse clock keeps the hot path cheapest. Either is acceptable. Please do NOT add a Close() here: middleware constructors only return fiber.Handler and there is no lifecycle hook that would ever call it, so a Close() would be dead code.

internal/storage/memory GC: this case is different. That Storage already has a real lifecycle (Close() exists), so keep the GC goroutine and make it cleanly stoppable instead of removing it: a buffered done channel plus an idempotent Close() that guards against double close and against sending to a goroutine that has already exited (this is exactly the deadlock called out as finding #5 in #4364). The fully lazy prune you have now means an idle store never reclaims expired entries until the next Get/Set, which is a semantic regression for the default storage.

Summary: coarse-clock singleton (or lazy) for logger/cache, stoppable goroutine (buffered done + idempotent Close) for the memory storage GC.

Copy link
Copy Markdown
Contributor Author

Copilot AI commented May 29, 2026

@copilot Thanks, this does remove the goroutine leak, but please reconsider the mechanism per case instead of removing all three goroutines uniformly.

Logger + cache timestamp: the original bug w...

Addressed in b7a14fa (plus 8cef4d3 for the follow-up field alignment). Logger/cache stay on the lazy request-time path, internal/memory was reverted, and internal/storage/memory now keeps its GC goroutine with a buffered done channel and idempotent Close() coverage.

Copilot AI changed the title 🐛 bug: remove unbounded background goroutines from logger, cache, and internal memory 🐛 bug: remove unbounded background goroutines from logger, cache, and internal storage memory May 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

🐛 [Bug]: Three goroutine leaks — background goroutines with no shutdown mechanism

2 participants