Skip to content

fix(offline): honor OFFLINE_WORK in git-ref2info and memoize TTL#9797

Open
iav wants to merge 1 commit into
mainfrom
fix/offline-work-respect-cache
Open

fix(offline): honor OFFLINE_WORK in git-ref2info and memoize TTL#9797
iav wants to merge 1 commit into
mainfrom
fix/offline-work-respect-cache

Conversation

@iav
Copy link
Copy Markdown
Contributor

@iav iav commented May 10, 2026

Summary

Fixes #6439.

memoized_git_ref_to_info() unconditionally ran git ls-remote against the upstream source even with OFFLINE_WORK=yes; when the remote was unreachable (e.g. git.kernel.org 502) the build aborted with a misleading network error instead of honoring the offline flag.

Two coordinated guards:

  • memoize-cached.sh: when OFFLINE_WORK=yes, serve the cache file regardless of TTL (the alternative is a network call that will fail). The "stale cache served" event is logged via display_alert.

  • git-ref2info.sh: early offline guard before the ls-remote loop. For ref_type=commit the SHA1 is taken from the ref itself (no network); for branch/tag we look up a pinned SHA1 in config/sources/git_sources.json. If neither is available, exit_with_error with a clear message instead of letting curl/ls-remote surface as 502.

    The same guard is applied to the include_makefile_body block (curl to git host) since there is no local-bare fallback in scope here.

Behaviour matrix unchanged for OFFLINE_WORK=no (default).

OFFLINE_WORK memoize cache pinned in git_sources.json result
no hit (fresh) * use cache (unchanged)
no hit (stale) * regen via ls-remote (unchanged)
no miss yes ls-remote then pinned override (unchanged)
no miss no ls-remote (unchanged)
yes hit (any age) * use cache (NEW: ignore TTL)
yes miss yes use pinned, no network (NEW)
yes miss no fail with clear message (NEW: previously misleading 502)
yes Makefile body needed + miss * fail with clear message (NEW)

Test plan

Verified via standalone scaffold against an isolated ${SRC}/cache/memoize/ sandbox (see PR comment for full output):

  • OFFLINE_WORK=yes build with prior populated cache → uses cache, no network calls
  • OFFLINE_WORK=yes build with empty cache + no pinned SHA1 → fails with clear "OFFLINE_WORK=yes but no SHA1 available ... run online once or pin sha1" message
  • OFFLINE_WORK=yes build with empty cache + pinned in config/sources/git_sources.json → uses pinned SHA1, no network calls
  • OFFLINE_WORK=no (default) build → behaviour unchanged (online ls-remote returned real SHA1)
  • OFFLINE_WORK=yes kernel-config (needs Makefile body) without cache → fails with clear "Makefile body ... not in cache" message

Summary by CodeRabbit

  • New Features

    • Offline build mode now uses pinned git refs from local config to resolve non-commit refs when offline.
    • Memoized cache handling can serve stale cached results in offline mode (with logging) so builds can continue without network access.
  • Bug Fixes

    • Improved fast-fail and clearer error messages when required Makefile bodies or pinned refs are missing from the cache.

Review Change Stack

@iav iav requested a review from a team as a code owner May 10, 2026 17:45
@iav iav requested review from lanefu and swissiety and removed request for a team May 10, 2026 17:45
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 10, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: bfdb1046-f671-40f8-8c4a-e9e4f55dba87

📥 Commits

Reviewing files that changed from the base of the PR and between c4cabdc and a7acb6f.

📒 Files selected for processing (2)
  • lib/functions/general/git-ref2info.sh
  • lib/functions/general/memoize-cached.sh

📝 Walkthrough

Walkthrough

Adds OFFLINE_WORK behavior: serve stale memoize cache when offline, use pinned SHAs from ${SRC}/config/sources/git_sources.json instead of git ls-remote for non-commit refs when offline, and fail fast when Makefile body is uncached in offline mode.

Changes

Offline Mode Support

Layer / File(s) Summary
Cache staleness in run_memoized
lib/functions/general/memoize-cached.sh
Centralizes staleness via cache_is_stale; when OFFLINE_WORK=yes a stale on-disk cache is kept and served (with a "using stale cache" log) instead of being deleted; flock retry/wait integers read from MEMOIZE_FLOCK_WAIT_INTERVAL/MEMOIZE_FLOCK_MAX_WAIT.
Git reference resolution with offline support
lib/functions/general/git-ref2info.sh
For non-commit refs, memoized_git_ref_to_info() looks up a pinned 40-hex SHA in ${SRC}/config/sources/git_sources.json when OFFLINE_WORK=yes and skips git ls-remote if found; if not found it exits with an explicit error. Adds an early-fail for Makefile-body fetch when offline and uncached.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

I’m a rabbit in the build tree, light and quick,
Serving stale caches when the network’s sick.
I nibble pinned SHAs from a JSON treat,
Skip remote calls and keep the build neat.
Hop on—offline builds now stay slick! 🐇✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 66.67% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'fix(offline): honor OFFLINE_WORK in git-ref2info and memoize TTL' accurately summarizes the main change: adding offline support to prevent network calls when OFFLINE_WORK=yes.
Linked Issues check ✅ Passed The PR fully addresses issue #6439 by implementing offline guards in git-ref2info.sh and memoize-cached.sh, preventing network calls when OFFLINE_WORK=yes and providing clear error messages with actionable guidance.
Out of Scope Changes check ✅ Passed All changes are directly scoped to the linked issue: modifications to git-ref2info.sh and memoize-cached.sh for OFFLINE_WORK support, with no unrelated alterations detected.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/offline-work-respect-cache

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions github-actions Bot added size/small PR with less then 50 lines 05 Milestone: Second quarter release Needs review Seeking for review Framework Framework components labels May 10, 2026
@iav
Copy link
Copy Markdown
Contributor Author

iav commented May 10, 2026

Test results

Standalone scaffold (.tmp/test-offline-pr9797.sh) drives run_memoized + memoized_git_ref_to_info against an isolated ${SRC}/cache/memoize/ sandbox.

# Scenario Result
1 online, cache miss Producing new & cachinggit ls-remote → real SHA1 of linux-7.0.y HEAD (3fd2ca34…)
2 offline, cache hit Using cached → same SHA1, no network
3 offline, cache miss, no pin fatal: OFFLINE_WORK=yes but no SHA1 available for '<source>' 'branch' 'linux-7.0.y' - run online once to populate cache, or pin sha1 in config/sources/git_sources.json
4 offline, cache miss, pinned in git_sources.json OFFLINE_WORK: using pinned SHA1 from git_sources.json → pinned SHA1 returned, no network
5 offline, cache miss, include_makefile_body fatal: OFFLINE_WORK=yes but Makefile body for '<source>' 'linux-7.0.y' (sha1 …) not in cache - run online once to populate <SRC>/cache/memoize/

All three guards exercised; OFFLINE_WORK=no path verified unchanged via scenario 1.

@igorpecovnik igorpecovnik added 08 Milestone: Third quarter release and removed 05 Milestone: Second quarter release labels May 16, 2026
@iav iav force-pushed the fix/offline-work-respect-cache branch from 88aabfd to accc536 Compare May 20, 2026 00:55
@github-actions github-actions Bot added the 05 Milestone: Second quarter release label May 20, 2026
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@lib/functions/general/git-ref2info.sh`:
- Around line 41-43: The offline_pinned assignment can produce multiple SHA
lines when git_sources.json has duplicate source+branch entries; change the jq
invocation in the offline_pinned assignment so it returns only a single match
(the first) for the given MEMO_DICT[GIT_SOURCE] and ref_name instead of multiple
lines (e.g., use jq’s first(...) or index [0] on the matching array) so the
subsequent 40-hex validation sees a single deterministic SHA for offline
resolution.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 9fa27bd7-cd6a-4c46-ade3-b86004f8cfba

📥 Commits

Reviewing files that changed from the base of the PR and between 88aabfd and accc536.

📒 Files selected for processing (2)
  • lib/functions/general/git-ref2info.sh
  • lib/functions/general/memoize-cached.sh

Comment thread lib/functions/general/git-ref2info.sh
@iav iav force-pushed the fix/offline-work-respect-cache branch from accc536 to c4cabdc Compare May 20, 2026 06:32
Fixes #6439.

memoized_git_ref_to_info() unconditionally ran 'git ls-remote' against the
upstream source even with OFFLINE_WORK=yes; when the remote was unreachable
the build aborted with a misleading network error.

Two coordinated guards:

* memoize-cached.sh: when OFFLINE_WORK=yes, serve the cache file regardless
  of TTL (alternative is a network call that will fail). The "stale cache
  served" event is logged.

* git-ref2info.sh: early offline guard before the ls-remote loop. For
  ref_type=commit the SHA1 is taken from the ref itself (no network); for
  branch/tag we look up a pinned SHA1 in config/sources/git_sources.json.
  If neither is available, exit_with_error with a clear message instead
  of letting curl/ls-remote surface as 502.

  Same guard applied to the include_makefile_body block (curl to git host)
  since there is no local-bare fallback in scope here.

Behaviour matrix unchanged for OFFLINE_WORK=no (default).

Assisted-by: Claude:claude-opus-4.7
@iav iav force-pushed the fix/offline-work-respect-cache branch from c4cabdc to a7acb6f Compare May 21, 2026 19:27
@iav
Copy link
Copy Markdown
Contributor Author

iav commented May 27, 2026

@ColorfulRhino ping

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

05 Milestone: Second quarter release 08 Milestone: Third quarter release Framework Framework components Needs review Seeking for review size/small PR with less then 50 lines

Development

Successfully merging this pull request may close these issues.

Scripts are still trying to fetch SHA1 online even if OFFLINE_WORK=yes is set

2 participants