Skip to content

Add configurable maximum number of attempts and wait time to download retries #820

@Piedone

Description

@Piedone

Is your feature request related to a problem? Please describe.

When downloading a package fails, LibMan can retry it, but it doesn't seem to do this when simply downloading a package anew. This causes sporadic build failures for us with the kenwheeler/slick@ecb6ea2 library references (and it seems only this one), LibMan failing with the following exception:

/home/runner/.nuget/packages/microsoft.web.librarymanager.build/3.0.71/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error : Microsoft.Web.LibraryManager.Contracts.ResourceDownloadException: Failed to download resource from "https://cdn.jsdelivr.net/gh/kenwheeler/slick@ecb6ea2/slick/slick.min.css" [/home/runner/work/Open-Source-Orchard-Core-Extensions/Open-Source-Orchard-Core-Extensions/src/Modules/Lombiq.UIKit/Lombiq.UIKit/Lombiq.UIKit.csproj]
/home/runner/.nuget/packages/microsoft.web.librarymanager.build/3.0.71/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error :  ---> System.Net.Http.HttpRequestException: Response status code does not indicate success: 404 (Not Found). [/home/runner/work/Open-Source-Orchard-Core-Extensions/Open-Source-Orchard-Core-Extensions/src/Modules/Lombiq.UIKit/Lombiq.UIKit/Lombiq.UIKit.csproj]
/home/runner/.nuget/packages/microsoft.web.librarymanager.build/3.0.71/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error :    at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode() [/home/runner/work/Open-Source-Orchard-Core-Extensions/Open-Source-Orchard-Core-Extensions/src/Modules/Lombiq.UIKit/Lombiq.UIKit/Lombiq.UIKit.csproj]
/home/runner/.nuget/packages/microsoft.web.librarymanager.build/3.0.71/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error :    at System.Net.Http.HttpClient.GetStreamAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken) [/home/runner/work/Open-Source-Orchard-Core-Extensions/Open-Source-Orchard-Core-Extensions/src/Modules/Lombiq.UIKit/Lombiq.UIKit/Lombiq.UIKit.csproj]
/home/runner/.nuget/packages/microsoft.web.librarymanager.build/3.0.71/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error :    at Microsoft.Web.LibraryManager.Cache.WebRequestHandler.GetStreamAsync(String url, CancellationToken cancellationToken) [/home/runner/work/Open-Source-Orchard-Core-Extensions/Open-Source-Orchard-Core-Extensions/src/Modules/Lombiq.UIKit/Lombiq.UIKit/Lombiq.UIKit.csproj]
/home/runner/.nuget/packages/microsoft.web.librarymanager.build/3.0.71/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error :    --- End of inner exception stack trace --- [/home/runner/work/Open-Source-Orchard-Core-Extensions/Open-Source-Orchard-Core-Extensions/src/Modules/Lombiq.UIKit/Lombiq.UIKit/Lombiq.UIKit.csproj]
/home/runner/.nuget/packages/microsoft.web.librarymanager.build/3.0.71/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error :    at Microsoft.Web.LibraryManager.Cache.WebRequestHandler.GetStreamAsync(String url, CancellationToken cancellationToken) [/home/runner/work/Open-Source-Orchard-Core-Extensions/Open-Source-Orchard-Core-Extensions/src/Modules/Lombiq.UIKit/Lombiq.UIKit/Lombiq.UIKit.csproj]
/home/runner/.nuget/packages/microsoft.web.librarymanager.build/3.0.71/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error :    at Microsoft.Web.LibraryManager.Cache.CacheService.DownloadToFileAsync(String url, String fileName, Int32 attempts, CancellationToken cancellationToken) [/home/runner/work/Open-Source-Orchard-Core-Extensions/Open-Source-Orchard-Core-Extensions/src/Modules/Lombiq.UIKit/Lombiq.UIKit/Lombiq.UIKit.csproj]
/home/runner/.nuget/packages/microsoft.web.librarymanager.build/3.0.71/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error :    at Microsoft.Web.LibraryManager.Cache.CacheService.<>c__DisplayClass9_0.<<RefreshCacheAsync>g__DownloadFileIfNecessaryAsync|0>d.MoveNext() [/home/runner/work/Open-Source-Orchard-Core-Extensions/Open-Source-Orchard-Core-Extensions/src/Modules/Lombiq.UIKit/Lombiq.UIKit/Lombiq.UIKit.csproj]
/home/runner/.nuget/packages/microsoft.web.librarymanager.build/3.0.71/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error : --- End of stack trace from previous location --- [/home/runner/work/Open-Source-Orchard-Core-Extensions/Open-Source-Orchard-Core-Extensions/src/Modules/Lombiq.UIKit/Lombiq.UIKit/Lombiq.UIKit.csproj]
/home/runner/.nuget/packages/microsoft.web.librarymanager.build/3.0.71/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error :    at Microsoft.Web.LibraryManager.Utilities.ParallelUtility.DoActionAndRelease[T](Func`2 act, T input, SemaphoreSlim s) [/home/runner/work/Open-Source-Orchard-Core-Extensions/Open-Source-Orchard-Core-Extensions/src/Modules/Lombiq.UIKit/Lombiq.UIKit/Lombiq.UIKit.csproj]
/home/runner/.nuget/packages/microsoft.web.librarymanager.build/3.0.71/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error :    at Microsoft.Web.LibraryManager.Utilities.ParallelUtility.ForEachAsync[T](Func`2 action, Int32 degreeOfParallelism, IEnumerable`1 items, CancellationToken cancellationToken) [/home/runner/work/Open-Source-Orchard-Core-Extensions/Open-Source-Orchard-Core-Extensions/src/Modules/Lombiq.UIKit/Lombiq.UIKit/Lombiq.UIKit.csproj]
/home/runner/.nuget/packages/microsoft.web.librarymanager.build/3.0.71/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error :    at Microsoft.Web.LibraryManager.Cache.CacheService.RefreshCacheAsync(IEnumerable`1 librariesCacheMetadata, ILogger logger, CancellationToken cancellationToken) [/home/runner/work/Open-Source-Orchard-Core-Extensions/Open-Source-Orchard-Core-Extensions/src/Modules/Lombiq.UIKit/Lombiq.UIKit/Lombiq.UIKit.csproj]
/home/runner/.nuget/packages/microsoft.web.librarymanager.build/3.0.71/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error :    at Microsoft.Web.LibraryManager.Providers.BaseProvider.RefreshCacheAsync(ILibraryInstallationState state, ILibrary library, CancellationToken cancellationToken) [/home/runner/work/Open-Source-Orchard-Core-Extensions/Open-Source-Orchard-Core-Extensions/src/Modules/Lombiq.UIKit/Lombiq.UIKit/Lombiq.UIKit.csproj]
libman.json : error LIB010: Failed to download resource from "https://cdn.jsdelivr.net/gh/kenwheeler/slick@ecb6ea2/slick/slick.min.css" [/home/runner/work/Open-Source-Orchard-Core-Extensions/Open-Source-Orchard-Core-Extensions/src/Modules/Lombiq.UIKit/Lombiq.UIKit/Lombiq.UIKit.csproj]

Describe the solution you'd like

Ideally, we would not get such download failures in the first place. However, this seems to be noticeably frequent for kenwheeler/slick@ecb6ea2 for some reason. Only jsdelivr supports such digest expressions, so we can't try the others.

However, if this can't be solved somehow for good, it'd be great to have better retries:

  • Make the number of attempts configurable, and use a default of 5, like it is for cache refreshes.
  • The current backoff time between attempts is 200ms. This is too low for transient errors to reliably clear (even with 5 attempts it would be just 1s). Instead, make this configurable (we'd use something like 5s).

Describe alternatives you've considered

Having fallback providers, but that brings its own challenges.

Additional context

Nothing else to add.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions