Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow nix realisation info to return partial results #12128

Open
2 tasks done
PkmX opened this issue Dec 31, 2024 · 4 comments
Open
2 tasks done

Allow nix realisation info to return partial results #12128

PkmX opened this issue Dec 31, 2024 · 4 comments
Labels
ca-derivations Derivations with content addressed outputs feature Feature request or proposal new-cli Relating to the "nix" command

Comments

@PkmX
Copy link

PkmX commented Dec 31, 2024

Is your feature request related to a problem?

We have a large number (~2K) of CA derivations, and our tool use nix realisation to mass check whether those derivations have been realised on the remote store.

However, nix realisation info will bail if any of the derivation is not realised on the remote store:

these 5 derivations will be built:
  /nix/store/<...>.drv
  /nix/store/<...>.drv
  /nix/store/<...>.drv
  /nix/store/<...>.drv
  /nix/store/<...>.drv
error: cannot operate on output 'out' of the unbuilt derivation '/nix/store/<...>.drv'

Proposed solution

Add an option --ignore-non-realised (name subject to change) to ignore the error and still output other realisations.

Alternative solutions

Run nix realisation info on each input derivation separately. This works but is really slow since there is a ton of duplicated work.

Additional context

Checklist


Add 👍 to issues you find important.

@PkmX PkmX added the feature Feature request or proposal label Dec 31, 2024
@roberth roberth added ca-derivations Derivations with content addressed outputs new-cli Relating to the "nix" command labels Dec 31, 2024
@Mic92
Copy link
Member

Mic92 commented Jan 7, 2025

Don't have a nix-native solution for you just now, but nix-eval-jobs --check-cache-status provides you this information as well, assuming you are using it for evaluation as well.

@PkmX
Copy link
Author

PkmX commented Jan 8, 2025

@Mic92 Does it work with CA derivations? Last time I tried it always returned local for cacheStatus.

@Mic92
Copy link
Member

Mic92 commented Jan 8, 2025

Unsure. We fixed some support for ca derivation, but I don't know if we have bugs in https://github.com/nix-community/nix-eval-jobs/pulls?q=is%3Apr+ca+is%3Aclosed that I have not discovered yet in cacheStatus. How can I test?

@PkmX
Copy link
Author

PkmX commented Jan 8, 2025

At the risk of getting off-topic, here are some simple tests:

$ nix run github:nix-community/nix-eval-jobs -- --expr 'derivation { name = "test"; builder = "/this/certainly/does/not/exist"; __contentAddressed = false; system = "x86_64-linux"; }' --check-cache-status | jq
{
  "attr": "",
  "attrPath": [],
  "cacheStatus": "notBuilt",
  "drvPath": "/nix/store/pjj51k74k6wvckpzp5kca7xgprn1xzjl-test.drv",
  "isCached": false,
  "name": "test",
  "neededBuilds": [],
  "neededSubstitutes": [],
  "outputs": {
    "out": "/nix/store/32yj8nmwjipcs18g88qf4q7kf80lbkfn-test"
  },
  "system": "x86_64-linux"
}
$ nix run github:nix-community/nix-eval-jobs -- --expr 'derivation { name = "test"; builder = "/this/certainly/does/not/exist"; __contentAddressed = true; system = "x86_64-linux"; }' --check-cache-status | jq
{
  "attr": "",
  "attrPath": [],
  "cacheStatus": "local",
  "drvPath": "/nix/store/hdprfagw99lzl7ypzzdmj482vh3g9gwj-test.drv",
  "isCached": true,
  "name": "test",
  "neededBuilds": [],
  "neededSubstitutes": [],
  "outputs": {
    "out": null
  },
  "system": "x86_64-linux"
}

I would expect that the CA derivation to also output notBuilt for cacheStatus.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ca-derivations Derivations with content addressed outputs feature Feature request or proposal new-cli Relating to the "nix" command
Projects
None yet
Development

No branches or pull requests

3 participants