TLDR;
It can happen that during an update phase of your NixOS installation some package error arise from a package that is not directly mentionned in your system configuration. This blog post presents 4 ways to find where a dependency comes from when you find yourself in this position.
The 4 ways described are:
- Relying on the
--show-traceoption ofnixos-rebuildto get a more verbose output. - Browsing the
nixpkgsrepository. - Using the
--query --referrersand--query --referencesoptions of thenix-storeutility. - Relying on the
why-dependscommand of thenixpackage manager.
The problem
The other day I was going to upgrade my NixOS installation and thus, after updating my flake lock file with nix flake update, I tried to run the upgrade using nixos-rebuild switch --use-remote-sudo --flake .. To my surprise, I was greeted with an error:
❯ nixos-rebuild switch --use-remote-sudo --flake .
building the system configuration...
error:
… while calling the 'head' builtin
at /nix/store/l0992b1shksi7k8w1x6wzi61zbs48vck-source/lib/attrsets.nix:1575:11:
1574| || pred here (elemAt values 1) (head values) then
1575| head values
| ^
1576| else
… while evaluating the attribute 'value'
at /nix/store/l0992b1shksi7k8w1x6wzi61zbs48vck-source/lib/modules.nix:809:9:
808| in warnDeprecation opt //
809| { value = builtins.addErrorContext "while evaluating the option `${showOption loc}':" value;
| ^
810| inherit (res.defsFinal') highestPrio;
… while evaluating the option `system.build.toplevel':
… while evaluating definitions from `/nix/store/l0992b1shksi7k8w1x6wzi61zbs48vck-source/nixos/modules/system/activation/top-level.nix':
… while evaluating the option `warnings':
… while evaluating definitions from `/nix/store/l0992b1shksi7k8w1x6wzi61zbs48vck-source/nixos/modules/system/boot/systemd.nix':
… while evaluating the option `systemd.services.home-manager-martin.serviceConfig':
… while evaluating definitions from `/nix/store/l0992b1shksi7k8w1x6wzi61zbs48vck-source/flake.nix':
… while evaluating the option `home-manager.users.martin.home.file.".manpath".source':
… while evaluating definitions from `/nix/store/hxhym8c5xz6dxkl3d9yppiwlnzk3khn7-source/modules/files.nix':
… while evaluating the option `home-manager.users.martin.home.file.".manpath".text':
… while evaluating definitions from `/nix/store/hxhym8c5xz6dxkl3d9yppiwlnzk3khn7-source/modules/programs/man.nix':
(stack trace truncated; use '--show-trace' to show the full, detailed trace)
error: Package ‘electron-27.3.11’ in /nix/store/4cpakzyvfw1rmm9v5i3387x6jd2h1v86-source/pkgs/development/tools/electron/binary/generic.nix:36 is marked as insecure, refusing to evaluate.
Known issues:
- Electron version 27.3.11 is EOL
You can install it anyway by allowing this package, using the
following methods:
a) To temporarily allow all insecure packages, you can use an environment
variable for a single invocation of the nix tools:
$ export NIXPKGS_ALLOW_INSECURE=1
Note: When using `nix shell`, `nix build`, `nix develop`, etc with a flake,
then pass `--impure` in order to allow use of environment variables.
b) for `nixos-rebuild` you can add ‘electron-27.3.11’ to
`nixpkgs.config.permittedInsecurePackages` in the configuration.nix,
like so:
{
nixpkgs.config.permittedInsecurePackages = [
"electron-27.3.11"
];
}
c) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
‘electron-27.3.11’ to `permittedInsecurePackages` in
~/.config/nixpkgs/config.nix, like so:
{
permittedInsecurePackages = [
"electron-27.3.11"
];
}
Apparently, my NixOS configuration uses Electron in an EOL version, namely 27.3.11, and it is asking me to explicitly declare that I allow this package to be installed although its support phase would mark it as insecure. Okay, thank you Nix for the heads up! Problem is, I never explicitly asked for Electron to be installed:
~/nix-configuration main*
❯ rg -i electron
~/nix-configuration main*
❯
So, where does this come from? This clearly seems to be a dependency of my NixOS configuration and we will now explore how we can figure out the culprit.
The solutions
Asking for more trace
The first approach would simply be to ask a more verbose output from the nixos-rebuild utility and investigate from there. In order to do so, we can simply use the --show-trace option and we instantly get a lot more details. I won’t paste the whole output because it is a lot, but here is what we get before ending with the same message than before:
… while evaluating derivation 'logseq-0.10.9'
whose name attribute is located at /nix/store/4cpakzyvfw1rmm9v5i3387x6jd2h1v86-source/pkgs/stdenv/generic/make-derivation.nix:334:7
… while evaluating attribute 'postFixup' of derivation 'logseq-0.10.9'
at /nix/store/4cpakzyvfw1rmm9v5i3387x6jd2h1v86-source/pkgs/by-name/lo/logseq/package.nix:62:3:
61|
62| postFixup = ''
| ^
63| # set the env "LOCAL_GIT_DIRECTORY" for dugite so that we can use the git in nixpkgs
… in the condition of the assert statement
at /nix/store/4cpakzyvfw1rmm9v5i3387x6jd2h1v86-source/lib/customisation.nix:366:17:
365| drvPath = assert condition; drv.drvPath;
366| outPath = assert condition; drv.outPath;
| ^
367| };
… while evaluating the attribute 'handled'
at /nix/store/4cpakzyvfw1rmm9v5i3387x6jd2h1v86-source/pkgs/stdenv/generic/check-meta.nix:507:7:
506| # or, alternatively, just output a warning message.
507| handled =
| ^
508| (
… from call site
at /nix/store/4cpakzyvfw1rmm9v5i3387x6jd2h1v86-source/pkgs/stdenv/generic/check-meta.nix:511:13:
510| else if valid == "no" then (
511| handleEvalIssue { inherit meta attrs; } { inherit (validity) reason errormsg; }
| ^
512| )
… while calling 'handleEvalIssue'
at /nix/store/4cpakzyvfw1rmm9v5i3387x6jd2h1v86-source/pkgs/stdenv/generic/check-meta.nix:269:38:
268|
269| handleEvalIssue = { meta, attrs }: { reason , errormsg ? "" }:
| ^
270| let
… while calling the 'throw' builtin
at /nix/store/4cpakzyvfw1rmm9v5i3387x6jd2h1v86-source/pkgs/stdenv/generic/check-meta.nix:281:8:
280| else throw;
281| in handler msg;
| ^
282|
error: Package ‘electron-27.3.11’ in /nix/store/4cpakzyvfw1rmm9v5i3387x6jd2h1v86-source/pkgs/development/tools/electron/binary/generic.nix:36 is marked as insecure, refusing to evaluate.
Known issues:
- Electron version 27.3.11 is EOL
You can install it anyway by allowing this package, using the
following methods:
a) To temporarily allow all insecure packages, you can use an environment
variable for a single invocation of the nix tools:
$ export NIXPKGS_ALLOW_INSECURE=1
Note: When using `nix shell`, `nix build`, `nix develop`, etc with a flake,
then pass `--impure` in order to allow use of environment variables.
b) for `nixos-rebuild` you can add ‘electron-27.3.11’ to
`nixpkgs.config.permittedInsecurePackages` in the configuration.nix,
like so:
{
nixpkgs.config.permittedInsecurePackages = [
"electron-27.3.11"
];
}
c) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
‘electron-27.3.11’ to `permittedInsecurePackages` in
~/.config/nixpkgs/config.nix, like so:
{
permittedInsecurePackages = [
"electron-27.3.11"
];
}
We can see that leading up to the error is the evaluation of the derivation logseq-0.10.9 and this is definitely something I have in my NixOS configuration. Okay, we got the culprit and if I remove it from the configuration it now works, but let’s continue to explore the other ways we could have found it.
Going directly for the nixpkgs repository
One convenient thing we could leverage on is that the whole Nix packages collection can be found on Github at https://github.com/NixOS/nixpkgs/. If we clone it locally and search it for electron_27 with ripgrep for instance, we can quickly end up with something interesting:
❯ rg -C 3 electron_27
pkgs/top-level/all-packages.nix
5649-
5650- logseq = callPackage ../by-name/lo/logseq/package.nix {
5651- # electron version from: https://github.com/logseq/logseq/blob/0.10.9/package.json#L116
5652: electron = electron_27;
5653- };
5654-
5655- long-shebang = callPackage ../misc/long-shebang { };
--
5753- };
5754-
5755- micropad = callPackage ../applications/office/micropad {
5756: electron = electron_27;
5757- };
5758-
5759- midicsv = callPackage ../tools/audio/midicsv { };
--
5984- pn = callPackage ../tools/text/pn { };
5985-
5986- pocket-casts = callPackage ../applications/audio/pocket-casts {
5987: electron = electron_27;
5988- };
5989-
5990- portal = callPackage ../tools/misc/portal { };
--
17724-
17725- inherit (callPackages ../development/tools/electron/binary { })
17726- electron_24-bin
17727: electron_27-bin
17728- electron_28-bin
17729- electron_29-bin
17730- electron_30-bin;
--
17735- electron-chromedriver_31;
17736-
17737- electron_24 = electron_24-bin;
17738: electron_27 = electron_27-bin;
17739- electron_28 = electron_28-bin;
17740- electron_29 = if lib.meta.availableOn stdenv.hostPlatform electron-source.electron_29 then electron-source.electron_29 else electron_29-bin;
17741- electron_30 = if lib.meta.availableOn stdenv.hostPlatform electron-source.electron_30 then electron-source.electron_30 else electron_30-bin;
--
34931- wgnord = callPackage ../applications/networking/wgnord/default.nix { };
34932-
34933- whalebird = callPackage ../applications/misc/whalebird {
34934: electron = electron_27;
34935- };
34936-
34937- windowlab = callPackage ../applications/window-managers/windowlab { };
There are only 4 packages in Nixpkgs that actually have this dependency, 3 of which I do not use:
- logseq
- micropad
- pocket-casts
- whalebird
In this instance, searching through the repository itself was a viable option because I knew I was searching for a specific package as a dependency. But sometimes your errors are less obvious than that or the dependency you are looking for is simply not as direct as it is here. Let’s get to the next option.
Interrogating the nix-store
A convenient aspect of NixOS is that we can build the system, test it and revert back very easily. On top of that it also offers ways to simply test the config without impacting your current system. Here we will explore how we can end up with a system loaded with our current “faulty” configuration and query the system to understand where this dependency issue comes from.
Our previous error messages already gave us ways to bypass the insecure package issue. In order to allow those packages one of the most straightforward solution is simply to add the NIXPKGS_ALLOW_INSECURE=1 environment variable to our command. This is a temporary solution and more persistent solutions are already described in our error message, but for debugging purposes this will be more than enough. The other important thing is that we have to specify the --impure option in order to tell Nix to take this variable into account, otherwise it will simply ignore the environment on build.
Now that we know how to build, let’s see the two options we could have at hand here. nixos-rebuild offers two interesting options, one is test that will leave you with your system in the state described in your NixOS configuration files but without persistence of it, meaning that on reboot you would end up with your current system. If we wanted to build the system this way we would then issue a NIXPKGS_ALLOW_INSECURE=1 nixos-rebuild test --use-remote-sudo --impure --flake .. The other option that we could consider is that instead of applying thit change on our currently running system, we could ask NixOS to instead build a VM to test it inside. In order to do so all you have to do is to use the build-vm option instead. This option will build a ready to launch VM, leaving you with a link in the result directory to start it: NIXPKGS_ALLOW_INSECURE=1 nixos-rebuild build-vm --use-remote-sudo --impure --flake .:
building the system configuration...
warning: Git tree '/home/martin/nix-configuration' is dirty
Done. The virtual machine can be started by running /nix/store/rfxw0ixvcxqkfxznaka29zqss3zyvr4g-nixos-vm/bin/run-champignon-vm
Now we can actually run it with ./result/bin/run-champignon-vm and then start querying its Nix store. The first thing we would like to know is, once installed, where is actually located our electron_27 package. You could search this directly inside your /nix/store folder for instance. One can issue a ls /nix/store | rg electron and get all electron related files, the issue being that you could have multiple instances of such files depending on the number of generations you have on your system; so how could you know which one is the one you are looking for? For that we can actually ask directly Nix using nix eval on our Flake file. For instance if we run NIXPKGS_ALLOW_INSECURE=1 nix eval --raw --impure .#nixosConfigurations.champignon.pkgs.electron_27 we get this as an output:
/nix/store/37n65w3vx3g1nfa76x497iv9c8cmg6zh-electron-27.3.11
Then, we simply ask questions to our nix-store. For instance, who is referencing this package with nix-store --query --referrers /nix/store/37n65w3vx3g1nfa76x497iv9c8cmg6zh-electron-27.3.11:
/nix/store/37n65w3vx3g1nfa76x497iv9c8cmg6zh-electron-27.3.11
/nix/store/cka877vw03qs9wsw9hm7cc5yyysazmba-logseq-0.10.9
/nix/store/b12k4z007wwj4pz42ip0pxjm351al05z-closure-info
/nix/store/l53sg82qflkqj4vhf67xlslanx2jzp32-closure-info
There, once again, we find back our culprit: logseq-0.10.9.
Asking nix why-depends
We know how to query the Nix store, but, what if we could just evaluate a generation and get this answer directly, without having to switch to this generation or build a vm to test it? For that we can rely on the nix utility with the command why-depends by passing it the configuration of your flake with .#nixosConfigurations.champignon.config.system.build.toplevel and the dependency, in our case .#nixosConfigurations.champignon.pkgs.electron_27. If you don’t know exactly where is the package you are looking for inside the flake structure, you can use nix repl and then load your flake file with :lf .; then you can navigate through the structure with the <tab> key for completion to look around.
Let’s see what happens now. We will still be using NIXPKGS_ALLOW_INSECURE=1 and --impure as they are needed for Nix to evaluate the flake:
❯ NIXPKGS_ALLOW_INSECURE=1 nix why-depends .#nixosConfigurations.champignon.config.system.build.toplevel .#nixosConfigurations.champignon.pkgs.electron_27 --impure
/nix/store/wpdjjrxxmyvx8l1m4hfqc4ybqz3ki3x2-nixos-system-champignon-24.05.20240804.8b5b672
└───/nix/store/0rh58q2x1d766skmyzv72y92bq366shx-etc
└───/nix/store/cm9cc21c8lhjqal6c5fq7gf88mc39b3s-user-environment
└───/nix/store/hyslf7m9ywdlm7pf6ipffzvmnc7wf7gw-home-manager-path
└───/nix/store/cka877vw03qs9wsw9hm7cc5yyysazmba-logseq-0.10.9
└───/nix/store/37n65w3vx3g1nfa76x497iv9c8cmg6zh-electron-27.3.11
How convenient, we directly end up with our well-known culprit: logseq-0.10.9.
Conclusion
I guess it’s time for a little wrap up. We’ve been through 4 ways to isolate the problematic package in our generation and if you were in this kind of trouble I hope I could help you. My personal preference goes to the last one, using why-depends as it is in my opinion the most straight to the point although you won’t hurt yourself with a small --show-trace to get more logs at first. As this is not a fabricated scenario but one I really encountered, I’d like to thank everyone who helped me solving this, especially on those two threads: