Release v0.3.3

Cache AUR package providers #23

README.md

@@ -2,6 +2,8 @@
 
 Decman is a declarative package & configuration manager for Arch Linux. It allows you to manage installed packages, your dotfiles, enabled systemd units, and run commands automatically. Your system is configured using python so your configuration can be very adaptive.
 
+If you want, you can also use decman with other configuration languages. See the [example with TOML later in this README](#decman-with-other-configuration-languages).
+
 ## Overview
 
 A complete example is available in the `example`-directory of this repository. It also serves as documentation so reading it is recommended.
@@ -78,10 +80,12 @@ Then import your module in your main source file.
 import decman
 from syncthing import Syncthing
 
+# NOTE! Removing a enabled module from decman.module means that on_disable will not run.
+# Instead disable the module.
 decman.modules += [Syncthing()]
 ```
 
-Then run decman.
+Then run decman. Note that terminal colors cannot be disabled for decman.
 
 > [!WARNING]
 > Decman runs as root. This means that your `source.py` will be executed as root as well.
@@ -102,6 +106,206 @@ Decman has some CLI options, to see them all run:
 decman --help
 ```
 
+## Installation
+
+Clone the decman PKGBUILD:
+
+```sh
+git clone https://github.com/kiviktnm/decman-pkgbuild.git
+```
+
+Review the PKGBUILD and install it.
+
+```sh
+cd decman-pkgbuild
+makepkg -si
+```
+
+So far I have not created an AUR package for decman, because I'm not sure if other people would find decman useful.
+
+## What decman manages?
+
+### Packages
+
+Decman can be used to install pacman packages. Decman will install all packages defined in the source and **remove** all explicitly installed packages not defined in the source. You don't need to list dependencies as those will be handeled by pacman. You can set packages to be ignored by decman, so that it won't install them nor remove them.
+
+```py
+# Include both foreign and pacman packages here.
+decman.ignored_packages += ["yay", "opendoas"]
+```
+
+### Foreign packages
+
+> [!NOTE]
+> Building of foreign packages is not the primary function of decman. There are some issues that I may or may not fix.
+> If you can't build a package using decman, consider adding it to `ignored_packages` and building it yourself.
+
+Decman can install AUR packages as well as user defined packages. Foreign packages are AUR and user packages combined.
+
+Here is an example of a user package. Managing user packages is somewhat cumbersome as you have to declare their versions, dependencies and make dependencies manually. However, you probably won't install many user packages anyway.
+
+```py
+decman.user_packages.append(
+    decman.UserPackage(
+        pkgname="decman-git",
+        # Note, this example may not be up to date
+        provides=["decman"],
+        version="0.3.3",
+        dependencies=["python", "python-requests", "devtools", "pacman", "systemd", "git"],
+        make_dependencies=[
+            "python-setuptools", "python-build", "python-installer", "python-wheel"
+        ],
+        git_url="https://github.com/kiviktnm/decman-pkgbuild.git",
+    ))
+```
+
+Building of foreign packages happens in a chroot. This creates some overhead, but ensures clean builds. By default the chroot is created to `/tmp/decman/build`. I recommend to use a tmpfs for the `/tmp/` directory to speed up builds. Also make sure that the tmpfs-partition is large enough. I recommend at least 6 GB.
+
+Build packages are stored in a cache `/var/cache/decman`. By default decman keeps 3 most recent versions of all packages.
+
+### Systemd units
+
+> [!NOTE]
+> Decman will only enable and disable systemd services. It will not start or stop them.
+
+Decman can enable systemd services, system wide or for a specific user. Decman will enable all units defined in the source, and disable them when they are removed from the source. If a unit is not defined in the source, decman will not touch it.
+
+### Files
+
+Decman functions as a dotfile manager. It will install the defined files and directories to their destinations. You can set file permissions, owners as well as define variables that will be substituted in the installed files. Decman keeps track of all files it creates and when a file is no longer present in your source, it will be also removed from its destination. This helps with keeping your system clean. However, decman won't ever remove directories as they might contain files that weren't created by decman.
+
+### Commands
+
+Modules have 4 methods: `on_enable`, `on_disable`, `after_update` and `after_version_change`. These will be executed if the module is enabled, the module is disabled, after every update and after the version of the module has changed. You can use the helper functions `prg` and `sh` to run programs. These programs could for example be used to update packages managed by another package manager.
+
+## Order of operations
+
+When decman runs, it does the following things in this order.
+
+1. Disable systemd units that are no longer in the source.
+1. Create and update files.
+1. Remove files no longer in the source.
+1. Remove packages not defined in the source.
+1. Upgrade packages.
+   - To upgrade foreign devel packages (eg. `*-git`) use the `--upgrade-devel` CLI option.
+1. Install new packages.
+1. Enable new systemd units.
+1. Run commands:
+   1. `on_enable`
+   1. `after_version_change`
+   1. `on_disable`
+   1. `after_update`
+
+Operations may be skipped with command line options.
+
+## Decman with other configuration languages
+
+Since decman uses Python files to declare your system, you can easily parse another configuration language in your Python source instead of declaring things directly in Python.
+
+<details>
+<summary>Here is a basic example using TOML.</summary>
+
+To use TOML with Python, you need the `toml`-package. To install it in Arch Linux, install the `python-toml`-package.
+
+This example doesn't allow you to use decman's modules or set decman's settings using TOML. You'll have to set them using Python. With this example you can use both TOML and Python if you want.
+It would be possible to add support for TOML decman modules, but I don't think it would be worth the effort.
+
+Write this in your decman source.
+
+```py
+import toml
+import decman
+
+TOML_CONFIG_FILE="/your/file/here.toml"
+
+# These functions convert TOML tables to decman Files/Directories/UserPackages.
+
+def toml_to_decman_file(toml_dict) -> decman.File:
+    return decman.File(content=toml_dict.get("content"),
+                       source_file=toml_dict.get("source_file"),
+                       bin_file=toml_dict.get("bin_file", False),
+                       encoding=toml_dict.get("encoding", "utf-8"),
+                       owner=toml_dict.get("owner"),
+                       group=toml_dict.get("group"),
+                       permissions=toml_dict.get("permissions", 0o644))
+
+
+def toml_to_decman_directory(toml_dict) -> decman.Directory:
+    return decman.Directory(source_directory=toml_dict["source_directory"],
+                            bin_files=toml_dict.get("bin_files", False),
+                            encoding=toml_dict.get("encoding", "utf-8"),
+                            owner=toml_dict.get("owner"),
+                            group=toml_dict.get("group"),
+                            permissions=toml_dict.get("permissions", 0o644))
+
+def toml_to_decman_user_package(toml_dict) -> decman.UserPackage:
+    return decman.UserPackage(pkgname=toml_dict["pkgname"],
+                              version=toml_dict["version"],
+                              dependencies=toml_dict["dependencies"],
+                              git_url=toml_dict["git_url"],
+                              pkgbase=toml_dict.get("pkgbase"),
+                              provides=toml_dict.get("provides"),
+                              make_dependencies=toml_dict.get("make_dependencies"),
+                              check_dependencies=toml_dict.get("check_dependencies"))
+
+
+# Parse TOML into a Python dictionary
+toml_source = toml.load(TOML_CONFIG_FILE)
+
+# Set decman variables using the parsed dictionary.
+
+decman.packages = toml_source.get("packages", [])
+decman.aur_packages = toml_source.get("aur_packages", [])
+decman.ignored_packages = toml_source.get("ignored_packages", [])
+decman.enabled_systemd_units = toml_source.get("enabled_systemd_units", [])
+decman.enabled_systemd_user_units = toml_source.get("enabled_systemd_user_units", {})
+
+for filename, toml_file_dec in toml_source.get("files", {}).items():
+    decman.files[filename] = toml_to_decman_file(toml_file_dec)
+
+for dirname, toml_dir_dec in toml_source.get("directories", {}).items():
+    decman.directories[dirname] = toml_to_decman_directory(toml_dir_dec)
+
+for toml_user_package_dec in toml_source.get("user_packages", []):
+    decman.user_packages.append(toml_to_decman_user_package(toml_user_package_dec))
+```
+
+Then you can use TOML configuration like this:
+
+```toml
+packages = ["python", "git", "networkmanager", "ufw", "neovim", "python-toml"]
+aur_packages = ["protonvpn"]
+enabled_systemd_units = ["NetworkManager.service"]
+ignored_packages = ["yay"]
+user_packages = [{
+    pkgname="decman-git",
+    # Note, this example may not be up to date
+    provides=["decman"],
+    version="0.3.3",
+    dependencies=["python", "python-requests", "devtools", "pacman", "systemd", "git"],
+    make_dependencies=[
+        "python-setuptools", "python-build", "python-installer", "python-wheel"
+    ],
+    git_url="https://github.com/kiviktnm/decman-pkgbuild.git"
+}]
+
+[files]
+'/etc/vconsole.conf' = { content="KEYMAP=us" }
+'/etc/pacman.conf' = { source_file="./dotfiles/pacman.conf" }
+
+[directories]
+'/home/user/.config/nvim' = { source_directory="./dotfiles/nvim", owner="user" }
+
+# To set other file/directory attributes, just add them like this. Note that here I am not using octal notation for permissions.
+'/home/user/.config/example' = { source_directory="./dotfiles/example", owner="user", group="user", bin_files=true, encoding="utf-8", permissions=448 }
+
+[enabled_systemd_user_units]
+user = ["syncthing.service"]
+user2 = ["example.service", "another.service"]
+```
+
+</details>
+
 ## Why use decman?
 
 Here are some reasons why I created decman for myself.
@@ -183,93 +387,6 @@ I tried NixOS in the past, but it had some issues that caused me to create decma
 - NixOS is hard, and the documentation (when I last tried it) wasn't that good. Doing more complex stuff was sometimes just very annoying.
 - NixOS has unnecessary abstraction with NixOS options. They are great until you have to configure something specific and there is not an option for it. Then you'll have to inline other configuration language within your Nix config. And if some software doesn't have any premade options you'll have to do write the config manually. Then you'll have some software managed with just options and others with normal config files. I prefer to keep everything consistent.
 
-## Installation
-
-Clone the decman PKGBUILD:
-
-```sh
-git clone https://github.com/kiviktnm/decman-pkgbuild.git
-```
-
-Review the PKGBUILD and install it.
-
-```sh
-cd decman-pkgbuild
-makepkg -si
-```
-
-So far I have not created an AUR package for decman, because I'm not sure if other people would find decman useful.
-
-## What decman manages?
-
-### Packages
-
-Decman can be used to install pacman packages. Decman will install all packages defined in the source and **remove** all packages not defined in the source. You can set packages to be ignored by decman, so that it won't install them nor remove them.
-
-```py
-# Include both foreign and pacman packages here.
-decman.ignored_packages += ["yay", "opendoas"]
-```
-
-### Foreign packages
-
-> [!NOTE]
-> Building of foreign packages is not the primary function of decman. There are some issues that I may or may not fix.
-> If you can't build a package using decman, consider adding it to `ignored_packages` and building it yourself.
-
-Decman can install AUR packages as well as user defined packages. Foreign packages are AUR and user packages combined.
-
-Here is an example of a user package. Managing user packages is somewhat cumbersome as you have to declare their versions, dependencies and make dependencies manually. However, you probably won't install many user packages anyway.
-
-```py
-decman.user_packages.append(
-    decman.UserPackage(
-        pkgname="decman-git",
-        # Note, this example may not be up to date
-        provides=["decman"],
-        version="0.1.0",
-        dependencies=["python", "python-requests", "devtools", "pacman", "systemd", "git"],
-        make_dependencies=[
-            "python-setuptools", "python-build", "python-installer", "python-wheel"
-        ],
-        git_url="https://github.com/kiviktnm/decman-pkgbuild.git",
-    ))
-```
-
-Building of foreign packages happens in a chroot. This creates some overhead, but ensures clean builds. Build packages are stored in a cache `/var/cache/decman`. By default decman keeps 3 most recent versions of all packages.
-
-### Systemd units
-
-Decman can enable systemd services, system wide or for a specific user. Decman will enable all units defined in the source, and disable them when they are removed from the source. If a unit is not defined in the source, decman will not touch it.
-
-### Files
-
-Decman functions as a dotfile manager. It will install the defined files and directories to their destinations. You can set file permissions, owners as well as define variables that will be substituted in the installed files. Decman keeps track of all files it creates and when a file is no longer present in your source, it will be also removed from its destination. This helps with keeping your system clean. However, decman won't ever remove directories as they might contain files that weren't created by decman.
-
-### Commands
-
-Modules have 4 methods: `on_enable`, `on_disable`, `after_update` and `after_version_change`. These will be executed if the module is enabled, the module is disabled, after every update and after the version of the module has changed. You can use the helper functions `prg` and `sh` to run programs. These programs could for example be used to update packages managed by another package manager.
-
-## Order of operations
-
-When decman runs, it does the following things in this order.
-
-1. Disable systemd units that are no longer in the source.
-1. Create and update files.
-1. Remove files no longer in the source.
-1. Remove packages not defined in the source.
-1. Upgrade packages.
-   - To upgrade foreign devel packages (eg. `*-git`) use the `--upgrade-devel` CLI option.
-1. Install new packages.
-1. Enable new systemd units.
-1. Run commands:
-   1. `on_enable`
-   1. `after_version_change`
-   1. `on_disable`
-   1. `after_update`
-
-Operations may be skipped with command line options.
-
 ## License
 
 Copyright (C) 2024 Kivi Kaitaniemi

example/my_module.py

@@ -82,7 +82,7 @@ class MyModule(Module):
         return [
             UserPackage(
                 pkgname="decman-git",
-                version="0.1.0",
+                version="0.3.3",
                 provides=["decman"],
                 dependencies=[
                     "python",

example/source.py

@@ -4,7 +4,7 @@
 import socket
 import os
 
-# Remember: Do NOT use from imports for global variables
+# Note: Do NOT use from imports for global variables
 # BAD: from decman import packages/modules/etc
 import decman
 import decman.config
@@ -40,7 +40,7 @@ decman.config.makepkg_user = "kk"
 decman.user_packages.append(
     UserPackage(
         pkgname="decman-git",
-        version="0.1.0",
+        version="0.3.3",
         provides=["decman"],
         dependencies=[
             "python",
@@ -58,7 +58,8 @@ decman.user_packages.append(
             "python-wheel",
         ],
         git_url="https://github.com/kiviktnm/decman-pkgbuild.git",
-    ))
+    )
+)
 
 # Managing only packages with decman is not that interesting.
 # Decman also has really powerful ways of managing config files, scripts etc.
@@ -68,23 +69,25 @@ decman.user_packages.append(
 
 # Define file content inline.
 # Default text file encoding is utf-8 but it can be changed.
-decman.files["/etc/vconsole.conf"] = File(content="KEYMAP=us",
+decman.files["/etc/vconsole.conf"] = File(content="KEYMAP=us", encoding="utf-8")
-                                          encoding="utf-8")
 
 # Include file content from another file, set the file owner and permissions.
 # The source_file is relative to the directory where the main decman source.py is located.
 # By default, the file group is set to the group of the owner, but it can be overridden with the group argument.
 decman.files["/home/kk/.bin/user-script.sh"] = File(
-    source_file="files/user-script.sh", owner="kk", permissions=0o744)
+    source_file="files/user-script.sh", owner="kk", permissions=0o744
+)
 
 # Non-text files such as images can also be managed.
 decman.files["/home/kk/.background.png"] = File(
-    source_file="files/i-dont-actually-exist.png", bin_file=True, owner="kk")
+    source_file="files/i-dont-actually-exist.png", bin_file=True, owner="kk"
+)
 
 # If you need to install multiple files at once, use directories.
 # All files from the source directory will be copied recursively to the target.
 decman.directories["/home/kk/.config/app/"] = Directory(
-    source_directory="files/app-config", owner="kk")
+    source_directory="files/app-config", owner="kk"
+)
 
 # Decman has built in support for managing systemd units as well.
 # Decman will enable services declared here, and disable services removed from here.
@@ -95,8 +98,9 @@ decman.enabled_systemd_units += ["NetworkManager.service"]
 # You can manage units for users as well.
 
 # Ensure that previous user unit declarations aren't overwritten and they are initialized.
-decman.enabled_systemd_user_units[
+decman.enabled_systemd_user_units["kk"] = decman.enabled_systemd_user_units.get(
-    "kk"] = decman.enabled_systemd_user_units.get("kk", [])
+    "kk", []
+)
 # Add user unit.
 decman.enabled_systemd_user_units["kk"].append("syncthing.service")
 
@@ -123,14 +127,31 @@ decman.modules += [my_own_mod]
 # Configuring the behavior of decman is also done here.
 # These are the default values.
 
+# Note: you probably don't want to change these 2 settings and instead you'll want to to use the --debug CLI option.
 # Show debug output
 decman.config.debug_output = False
+# Suppress output of some commands that you probably don't want to see.
+decman.config.suppress_command_output = True
 
 # Make output less verbose. Summaries are still printed.
 decman.config.quiet_output = False
 
-# Suppress output of some commands that you probably don't want to see.
+# Decman captures pacman command output, and any line (and adjacent lines) that contains any of
-decman.config.suppress_command_output = True
+# the following keywords (case-insensetive) will be printed after the pacman command finishes.
+#
+# REMEMBER: You should still generally pay attention to pacman output
+# since these keywords may not catch all cases.
+decman.config.pacman_output_keywords = [
+    "pacsave",
+    "pacnew",
+    # Additional keywords can be:
+    # "warning",
+    # "error",
+    # "note",
+    # They might cause too many highlights however.
+]
+# If you don't want to print lines that contain keywords, set this to False
+decman.config.print_pacman_output_highlights = True
 
 # The user which builds aur and user packages.
 # decman.config.makepkg_user = "nobody" # This was set in a previous example. Let's not override it.
@@ -156,48 +177,49 @@ decman.config.number_of_packages_stored_in_cache = 3
 # Create a child class of the decman.config.Commands class and override methods.
 # These are the defaults.
 class MyCommands(decman.config.Commands):
-
     def list_pkgs(self) -> list[str]:
         return ["pacman", "-Qeq", "--color=never"]
 
     def list_foreign_pkgs_versioned(self) -> list[str]:
         return ["pacman", "-Qm", "--color=never"]
 
+    # --color=always is used in many commands since --color=auto results in no color.
+    # It seems a sensible default for me, since decman already uses color and it can't be disabled.
+
     def install_pkgs(self, pkgs: list[str]) -> list[str]:
-        return ["pacman", "-S", "--asexplicit"] + pkgs
+        return ["pacman", "-S", "--color=always", "--needed"] + pkgs
 
     def install_files(self, pkg_files: list[str]) -> list[str]:
-        return ["pacman", "-U", "--asdeps"] + pkg_files
+        return ["pacman", "-U", "--color=always", "--asdeps"] + pkg_files
 
     def set_as_explicitly_installed(self, pkgs: list[str]) -> list[str]:
         return ["pacman", "-D", "--asexplicit"] + pkgs
 
     def install_deps(self, deps: list[str]) -> list[str]:
-        return ["pacman", "-S", "--needed", "--asdeps"] + deps
+        return ["pacman", "-S", "--color=always", "--needed", "--asdeps"] + deps
 
     def is_installable(self, pkg: str) -> list[str]:
         return ["pacman", "-Sddp", pkg]
 
     def upgrade(self) -> list[str]:
-        return ["pacman", "-Syu"]
+        return ["pacman", "-Syu", "--color=always"]
 
     def remove(self, pkgs: list[str]) -> list[str]:
-        return ["pacman", "-Rs"] + pkgs
+        return ["pacman", "-Rs", "--color=always"] + pkgs
 
     def enable_units(self, units: list[str]) -> list[str]:
-        return ["systemctl", "enable", "--now", "--quiet"] + units
+        return ["systemctl", "enable"] + units
 
     def disable_units(self, units: list[str]) -> list[str]:
-        return ["systemctl", "disable", "--quiet"] + units
+        return ["systemctl", "disable"] + units
 
-    def enable_user_units(self, units: list[str]) -> list[str]:
+    def enable_user_units(self, units: list[str], user: str) -> list[str]:
-        return ["systemctl", "enable", "--now", "--quiet", "--user"] + units
+        return ["systemctl", "--user", "-M", f"{user}@", "enable"] + units
 
-    def disable_user_units(self, units: list[str]) -> list[str]:
+    def disable_user_units(self, units: list[str], user: str) -> list[str]:
-        return ["systemctl", "disable", "--quiet"] + units
+        return ["systemctl", "--user", "-M", f"{user}@", "disable"] + units
 
-    def compare_versions(self, installed_version: str,
+    def compare_versions(self, installed_version: str, new_version: str) -> list[str]:
-                         new_version: str) -> list[str]:
         return ["vercmp", installed_version, new_version]
 
     def git_clone(self, repo: str, dest: str) -> list[str]:
@@ -209,6 +231,9 @@ class MyCommands(decman.config.Commands):
     def git_get_commit_id(self) -> list[str]:
         return ["git", "rev-parse", "HEAD"]
 
+    def git_log_commit_ids(self) -> list[str]:
+        return ["git", "log", "--format=format:%H"]
+
     def review_file(self, file: str) -> list[str]:
         return ["less", file]
 
@@ -217,19 +242,31 @@ class MyCommands(decman.config.Commands):
 
     def install_chroot_packages(self, chroot_dir: str, packages: list[str]):
         return [
-            "arch-nspawn", chroot_dir, "pacman", "-S", "--needed",
+            "arch-nspawn",
-            "--noconfirm"
+            chroot_dir,
+            "pacman",
+            "-S",
+            "--needed",
+            "--noconfirm",
         ] + packages
 
+    def resolve_real_name(self, chroot_dir: str, pkg: str) -> list[str]:
+        return [
+            "arch-nspawn",
+            chroot_dir,
+            "pacman",
+            "-Sddp",
+            "--print-format=%n",
+            pkg,
+        ]
+
     def remove_chroot_packages(self, chroot_dir: str, packages: list[str]):
-        return ["arch-nspawn", chroot_dir, "pacman", "-Rsu", "--noconfirm"
+        return ["arch-nspawn", chroot_dir, "pacman", "-Rsu", "--noconfirm"] + packages
-                ] + packages
 
-    def make_chroot_pkg(self, chroot_wd_dir: str, user: str,
+    def make_chroot_pkg(
-                        pkgfiles_to_install: list[str]) -> list[str]:
+        self, chroot_wd_dir: str, user: str, pkgfiles_to_install: list[str]
-        makechrootpkg_cmd = [
+    ) -> list[str]:
-            "makechrootpkg", "-c", "-r", chroot_wd_dir, "-U", user
+        makechrootpkg_cmd = ["makechrootpkg", "-c", "-r", chroot_wd_dir, "-U", user]
-        ]
 
         for pkgfile in pkgfiles_to_install:
             makechrootpkg_cmd += ["-I", pkgfile]
@@ -246,12 +283,11 @@ decman.config.commands = MyCommands()
 
 
 class PikaurWrapperCommands(decman.config.Commands):
-
     def list_pkgs(self) -> list[str]:
         return ["pikaur", "-Qeq"]
 
     def install_pkgs(self, pkgs: list[str]) -> list[str]:
-        return ["pikaur", "-S", "--asexplicit"] + pkgs
+        return ["pikaur", "-S"] + pkgs
 
     def upgrade(self) -> list[str]:
         return ["pikaur", "-Syu"]

pyproject.toml

@@ -4,15 +4,21 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "decman"
-version = "0.1.0"
+version = "0.3.3"
 description = "Declarative package & configuration manager for Arch Linux."
 license = {file = "LICENSE"}
 authors = [
     {name = "Kivi Kaitaniemi"}
 ]
 dependencies = [
-    "requests"
+    "requests",
 ]
 
 [project.scripts]
 decman = "decman.app:main"
+
+[dependency-groups]
+dev = [
+    "isort>=5.13.2",
+    "ruff>=0.9.3",
+]

src/decman/__init__.py

@@ -2,12 +2,13 @@
 Module for writing system configurations for decman.
 """
 
-import typing
-import pwd
 import grp
-import shutil
 import os
+import pwd
+import shutil
 import subprocess
+import typing
+
 import decman.error
 
 
@@ -20,9 +21,11 @@ class UserRaisedError(Exception):
         super().__init__(message)
 
 
-def sh(sh_cmd: str,
+def sh(
+    sh_cmd: str,
     user: typing.Optional[str] = None,
-       env_overrides: typing.Optional[dict[str, str]] = None):
+    env_overrides: typing.Optional[dict[str, str]] = None,
+):
     """
     Shortcut for running a shell command.
     """
@@ -49,16 +52,20 @@ def sh(sh_cmd: str,
                 f"Running user defined shell command failed because the user {user} doesn't exist."
             ) from e
 
-        with subprocess.Popen(sh_cmd, shell=True, group=gid, user=uid,
+        with subprocess.Popen(
-                              env=env) as process:
+            sh_cmd, shell=True, group=gid, user=uid, env=env
+        ) as process:
             if process.wait() != 0:
                 raise decman.error.UserFacingError(
-                    f"Running user shell command '{sh_cmd}' as {user} failed.")
+                    f"Running user shell command '{sh_cmd}' as {user} failed."
+                )
 
 
-def prg(command: list[str],
+def prg(
+    command: list[str],
     user: typing.Optional[str] = None,
-        env_overrides: typing.Optional[dict[str, str]] = None):
+    env_overrides: typing.Optional[dict[str, str]] = None,
+):
     """
     Shortcut for running a program.
     """
@@ -74,7 +81,8 @@ def prg(command: list[str],
             subprocess.run(command, check=True, env=env)
         except subprocess.CalledProcessError as e:
             raise decman.error.UserFacingError(
-                f"Running user defined program '{command}' failed.") from e
+                f"Running user defined program '{command}' failed."
+            ) from e
     else:
         try:
             uid = pwd.getpwnam(user).pw_uid
@@ -84,11 +92,11 @@ def prg(command: list[str],
                 f"Running user defined program failed because the user {user} doesn't exist."
             ) from e
 
-        with subprocess.Popen(command, group=gid, user=uid,
+        with subprocess.Popen(command, group=gid, user=uid, env=env) as process:
-                              env=env) as process:
             if process.wait() != 0:
                 raise decman.error.UserFacingError(
-                    f"Running user program '{command}' as {user} failed.")
+                    f"Running user program '{command}' as {user} failed."
+                )
 
 
 class File:
@@ -127,9 +135,7 @@ class File:
         if group is not None:
             self.gid = grp.getgrnam(group).gr_gid
 
-    def copy_to(self,
+    def copy_to(self, target: str, variables: typing.Optional[dict[str, str]] = None):
-                target: str,
-                variables: typing.Optional[dict[str, str]] = None):
         """
         Copies the contents of this file to the target file.
         """
@@ -138,8 +144,9 @@ class File:
 
         target_directory = os.path.dirname(target)
 
-        def create_missing_dirs(dirct: str, uid: typing.Optional[int],
+        def create_missing_dirs(
-                                gid: typing.Optional[int]):
+            dirct: str, uid: typing.Optional[int], gid: typing.Optional[int]
+        ):
             if not os.path.isdir(dirct):
                 parent_dir = os.path.dirname(dirct)
                 if not os.path.isdir(parent_dir):
@@ -161,8 +168,7 @@ class File:
         os.chmod(target, self.permissions)
 
     def _write_content(self, target: str, variables: dict[str, str]):
-        if self.source_file is not None and (self.bin_file
+        if self.source_file is not None and (self.bin_file or len(variables) == 0):
-                                             or len(variables) == 0):
             shutil.copy(self.source_file, target)
         elif self.bin_file and self.content is not None:
             with open(target, "wb") as file:
@@ -177,7 +183,9 @@ class File:
             with open(target, "wt", encoding=self.encoding) as file:
                 file.write(content)
         else:
-            assert self.content is not None, "Content should be set since source_file was not set."
+            assert self.content is not None, (
+                "Content should be set since source_file was not set."
+            )
             content = self.content
             for var, value in variables.items():
                 content = content.replace(var, value)
@@ -220,7 +228,9 @@ class Directory:
     def copy_to(
         self,
         target_directory: str,
-            variables: typing.Optional[dict[str, str]] = None) -> list[str]:
+        variables: typing.Optional[dict[str, str]] = None,
+        only_print: bool = False,
+    ) -> list[str]:
         """
         Copies the files in this directory to the target directory.
 
@@ -233,15 +243,18 @@ class Directory:
             for src_dir, _, src_files in os.walk("."):
                 for src_file in src_files:
                     src_path = os.path.join(src_dir, src_file)
-                    file = File(source_file=src_path,
+                    file = File(
+                        source_file=src_path,
                         bin_file=self.bin_files,
                         encoding=self.encoding,
                         owner=self.owner,
                         group=self.group,
-                                permissions=self.permissions)
+                        permissions=self.permissions,
-                    target = os.path.normpath(
+                    )
-                        os.path.join(target_directory, src_path))
+                    target = os.path.normpath(os.path.join(target_directory, src_path))
                     created.append(target)
+
+                    if not only_print:
                         file.copy_to(target, variables)
         finally:
             os.chdir(original_wd)

src/decman/app.py

@@ -8,9 +8,9 @@ import sys
 import traceback
 
 import decman
+import decman.config as conf
 import decman.error as err
 import decman.lib as l
-import decman.config as conf
 from decman.lib import fpm
 
 
@@ -23,50 +23,62 @@ def main():
 
     parser = argparse.ArgumentParser(
         prog="decman",
-        description=
+        description="Declarative package & configuration manager for Arch Linux",
-        "Declarative package & configuration manager for Arch Linux",
+        epilog="See more help at: https://github.com/kiviktnm/decman",
-        epilog="See more help at: https://github.com/kiviktnm/decman")
+    )
 
-    parser.add_argument("--source",
+    parser.add_argument(
-                        action="store",
+        "--source", action="store", help="python file containing configuration"
-                        help="python file containing configuration")
+    )
     parser.add_argument(
         "--print",
+        "--dry-run",
         action="store_true",
         default=False,
-        help=
+        help="print what would happen as a result of running decman",
-        "print what would happen as a result of running decman (doesn't print removed files)"
+    )
+    parser.add_argument(
+        "--debug", action="store_true", default=False, help="show debug output"
     )
     parser.add_argument(
         "--no-packages",
         action="store_true",
         default=False,
-        help="don't upgrade any packages (including foreign packages)")
+        help="don't upgrade any packages (including foreign packages)",
-    parser.add_argument("--no-foreign-packages",
+    )
+    parser.add_argument(
+        "--no-foreign-packages",
         action="store_true",
         default=False,
-                        help="don't upgrade foreign packages")
+        help="don't upgrade foreign packages",
-    parser.add_argument("--no-files",
+    )
+    parser.add_argument(
+        "--no-files", action="store_true", default=False, help="don't install any files"
+    )
+    parser.add_argument(
+        "--no-systemd-units",
         action="store_true",
         default=False,
-                        help="don't install any files")
+        help="don't enable/disable systemd units",
-    parser.add_argument("--no-systemd-units",
+    )
+    parser.add_argument(
+        "--no-commands",
         action="store_true",
         default=False,
-                        help="don't enable/disable systemd units")
+        help="don't run user specified commands",
-    parser.add_argument("--no-commands",
+    )
+    parser.add_argument(
+        "--upgrade-devel",
         action="store_true",
         default=False,
-                        help="don't run user specified commands")
+        help="upgrade devel packages",
-    parser.add_argument("--upgrade-devel",
+    )
-                        action="store_true",
-                        default=False,
-                        help="upgrade devel packages")
     parser.add_argument(
         "--force-build",
         action="store_true",
         default=False,
-        help="force building of packages that are already cached")
+        help="force building of packages that are already cached",
+    )
 
     args = parser.parse_args()
 
@@ -88,6 +100,13 @@ def main():
 
     try:
         opts = _set_up(store, args)
+        # Override debug_output if cli option is used
+        if args.debug:
+            conf.debug_output = True
+            conf.suppress_command_output = False
+        # When print cli option is used, show info output
+        if args.print:
+            conf.quiet_output = False
         Core(store, opts).run()
     except err.UserFacingError as error:
         l.print_error(error.user_facing_msg)
@@ -95,8 +114,7 @@ def main():
             l.print_debug(line)
         errored = True
     except decman.UserRaisedError as user_error:
-        l.print_error(
+        l.print_error(f"Error encountered while running the source: {user_error}")
-            f"Error encountered while running the source: {user_error}")
         errored = True
 
     # Save even when an error has occurred, since this avoids repeating steps like building pkgs.
@@ -148,13 +166,23 @@ def _set_up(store: l.Store, args):
             content = file.read()
     except OSError as e:
         raise err.UserFacingError(
-            f"Failed to read source file '{store.source_file}'.") from e
+            f"Failed to read source file '{store.source_file}'."
+        ) from e
 
     os.chdir(source_dir)
     sys.path.append(".")
     exec(content)
 
-    return args.print, not args.no_packages, not args.no_foreign_packages, not args.no_files, not args.no_systemd_units, not args.no_commands, args.upgrade_devel, args.force_build
+    return (
+        args.print,
+        not args.no_packages,
+        not args.no_foreign_packages,
+        not args.no_files,
+        not args.no_systemd_units,
+        not args.no_commands,
+        args.upgrade_devel,
+        args.force_build,
+    )
 
 
 class Core:
@@ -163,7 +191,16 @@ class Core:
     """
 
     def __init__(self, store: l.Store, opts):
-        self.only_print, self.update_packages, self.update_foreign_packages, self.update_files, self.update_units, self.run_commands, self.upgrade_devel, self.force_build = opts
+        (
+            self.only_print,
+            self.update_packages,
+            self.update_foreign_packages,
+            self.update_files,
+            self.update_units,
+            self.run_commands,
+            self.upgrade_devel,
+            self.force_build,
+        ) = opts
 
         self.store = store
         self.source = _resolve_source()
@@ -173,10 +210,10 @@ class Core:
 
         for upkg in self.source.all_user_pkgs():
             self.fpkg_search.add_user_pkg(
-                fpm.PackageInfo.from_user_package(upkg, self.pacman))
+                fpm.PackageInfo.from_user_package(upkg, self.pacman)
+            )
 
-        self.fpm = fpm.ForeignPackageManager(store, self.pacman,
+        self.fpm = fpm.ForeignPackageManager(store, self.pacman, self.fpkg_search)
-                                             self.fpkg_search)
 
     def run(self):
         """
@@ -207,20 +244,22 @@ class Core:
 
     def _disable_units(self):
         to_disable = self.source.units_to_disable(self.store)
-        l.print_list_summary("Disabling systemd units:", to_disable)
+        l.print_list("Disabling systemd units:", to_disable)
+        if to_disable:
+            l.print_info("Disabled systemd units won't be stopped automatically.")
         if not self.only_print:
             self.systemctl.disable_units(to_disable)
 
         user_units_to_disable = self.source.user_units_to_disable(self.store)
         for user, units in user_units_to_disable.items():
-            l.print_list_summary(f"Disabling systemd units for {user}:", units)
+            l.print_list(f"Disabling systemd units for {user}:", units)
             if not self.only_print:
                 self.systemctl.disable_user_units(units, user)
 
     def _remove_pkgs(self):
         currently_installed = self.pacman.get_installed()
         to_remove = self.source.packages_to_remove(currently_installed)
-        l.print_list_summary("Removing packages:", to_remove)
+        l.print_list("Removing packages:", to_remove)
         if not self.only_print:
             self.pacman.remove(to_remove)
 
@@ -229,22 +268,20 @@ class Core:
         if not self.only_print:
             self.pacman.upgrade()
             if conf.enable_fpm and self.update_foreign_packages:
-                self.fpm.upgrade(self.upgrade_devel, self.force_build,
+                self.fpm.upgrade(
-                                 self.source.ignored_packages)
+                    self.upgrade_devel, self.force_build, self.source.ignored_packages
+                )
 
     def _install_pkgs(self):
         currently_installed = self.pacman.get_installed()
-        to_install_pacman = self.source.pacman_packages_to_install(
+        to_install_pacman = self.source.pacman_packages_to_install(currently_installed)
-            currently_installed)
+        to_install_fpm = self.source.foreign_packages_to_install(currently_installed)
-        to_install_fpm = self.source.foreign_packages_to_install(
-            currently_installed)
 
-        l.print_list_summary("Installing pacman packages:", to_install_pacman)
+        l.print_list("Installing pacman packages:", to_install_pacman)
 
         # fpm prints a summary so no need to print it twice
         if self.only_print:
-            l.print_list_summary("Installing foreign packages:",
+            l.print_list("Installing foreign packages:", to_install_fpm)
-                                 to_install_fpm)
 
         if not self.only_print:
             self.pacman.install(to_install_pacman)
@@ -252,21 +289,17 @@ class Core:
                 self.fpm.install(to_install_fpm, force=self.force_build)
 
     def _create_and_remove_files(self):
-        l.print_list_summary("Installing files:",
+        l.print_summary("Installing files.")
-                             self.source.all_file_targets(),
+
-                             elements_per_line=1)
+        all_created = self.source.create_all_files(self.only_print)
-        l.print_list_summary("Installing directories:",
+        to_remove = self.source.files_to_remove(self.store, all_created)
-                             self.source.all_directory_targets(),
+
-                             elements_per_line=1)
+        l.print_list("Ensured files are up to date:", all_created, elements_per_line=1)
+        l.print_list("Removing files:", to_remove, elements_per_line=1)
 
         if self.only_print:
             return
 
-        all_created = self.source.create_all_files()
-        to_remove = self.source.files_to_remove(self.store, all_created)
-
-        l.print_list_summary("Removing files:", to_remove, elements_per_line=1)
-
         for file in to_remove:
             try:
                 os.remove(file)
@@ -278,13 +311,15 @@ class Core:
 
     def _enable_units(self):
         to_enable = self.source.units_to_enable(self.store)
-        l.print_list_summary("Enabling systemd units:", to_enable)
+        l.print_list("Enabling systemd units:", to_enable)
+        if to_enable:
+            l.print_info("Enabled systemd units won't be started automatically.")
         if not self.only_print:
             self.systemctl.enable_units(to_enable)
 
         user_units_to_enable = self.source.user_units_to_enable(self.store)
         for user, units in user_units_to_enable.items():
-            l.print_list_summary(f"Enabling systemd units for {user}:", units)
+            l.print_list(f"Enabling systemd units for {user}:", units)
             if not self.only_print:
                 self.systemctl.enable_user_units(units, user)
 

src/decman/config.py

@@ -45,26 +45,26 @@ class Commands:
         """
         Running this command installs the given packages from pacman repositories.
         """
-        return ["pacman", "-S", "--asexplicit"] + pkgs
+        return ["pacman", "-S", "--color=always", "--needed"] + pkgs
 
     def install_files(self, pkg_files: list[str]) -> list[str]:
         """
         Running this command installs the given packages files.
         """
-        return ["pacman", "-U", "--asdeps"] + pkg_files
+        return ["pacman", "-U", "--color=always", "--asdeps"] + pkg_files
 
     def set_as_explicitly_installed(self, pkgs: list[str]) -> list[str]:
         """
         Running this command installs sets the given as explicitly installed.
         """
-        return ["pacman", "-D", "--asexplicit"] + pkgs
+        return ["pacman", "-D", "--color=always", "--asexplicit"] + pkgs
 
     def install_deps(self, deps: list[str]) -> list[str]:
         """
         Running this command installs the given packages from pacman repositories.
         The packages are installed as dependencies.
         """
-        return ["pacman", "-S", "--needed", "--asdeps"] + deps
+        return ["pacman", "-S", "--color=always", "--needed", "--asdeps"] + deps
 
     def is_installable(self, pkg: str) -> list[str]:
         """
@@ -76,41 +76,40 @@ class Commands:
         """
         Running this command upgrades all pacman packages.
         """
-        return ["pacman", "-Syu"]
+        return ["pacman", "-Syu", "--color=always"]
 
     def remove(self, pkgs: list[str]) -> list[str]:
         """
         Running this command removes the given packages and their dependencies
         (that aren't required by other packages).
         """
-        return ["pacman", "-Rs"] + pkgs
+        return ["pacman", "-Rs", "--color=always"] + pkgs
 
     def enable_units(self, units: list[str]) -> list[str]:
         """
         Running this command enables the given systemd units.
         """
-        return ["systemctl", "enable", "--now", "--quiet"] + units
+        return ["systemctl", "enable"] + units
 
     def disable_units(self, units: list[str]) -> list[str]:
         """
         Running this command disables the given systemd units.
         """
-        return ["systemctl", "disable", "--quiet"] + units
+        return ["systemctl", "disable"] + units
 
-    def enable_user_units(self, units: list[str]) -> list[str]:
+    def enable_user_units(self, units: list[str], user: str) -> list[str]:
         """
-        Running this command enables the given systemd units for the user it's run as.
+        Running this command enables the given systemd units for the user.
         """
-        return ["systemctl", "enable", "--now", "--quiet", "--user"] + units
+        return ["systemctl", "--user", "-M", f"{user}@", "enable"] + units
 
-    def disable_user_units(self, units: list[str]) -> list[str]:
+    def disable_user_units(self, units: list[str], user: str) -> list[str]:
         """
-        Running this command disables the given systemd units fol the user it's run as.
+        Running this command disables the given systemd units for the user.
         """
-        return ["systemctl", "disable", "--quiet"] + units
+        return ["systemctl", "--user", "-M", f"{user}@", "disable"] + units
 
-    def compare_versions(self, installed_version: str,
+    def compare_versions(self, installed_version: str, new_version: str) -> list[str]:
-                         new_version: str) -> list[str]:
         """
         Running this command outputs -1 when the installed version is older than the new version.
         """
@@ -135,6 +134,12 @@ class Commands:
         """
         return ["git", "rev-parse", "HEAD"]
 
+    def git_log_commit_ids(self) -> list[str]:
+        """
+        Running this command outputs commit hashes of the repository.
+        """
+        return ["git", "log", "--format=format:%H"]
+
     def review_file(self, file: str) -> list[str]:
         """
         Running this command outputs a file for the user to see.
@@ -153,27 +158,42 @@ class Commands:
         Running this command installs the given packages to the given chroot.
         """
         return [
-            "arch-nspawn", chroot_dir, "pacman", "-S", "--needed",
+            "arch-nspawn",
-            "--noconfirm"
+            chroot_dir,
+            "pacman",
+            "-S",
+            "--needed",
+            "--noconfirm",
         ] + packages
 
+    def resolve_real_name(self, chroot_dir: str, pkg: str) -> list[str]:
+        """
+        This command prints a real name of a package. For example, it prints the package which provides a virtual package.
+        """
+        return [
+            "arch-nspawn",
+            chroot_dir,
+            "pacman",
+            "-Sddp",
+            "--print-format=%n",
+            pkg,
+        ]
+
     def remove_chroot_packages(self, chroot_dir: str, packages: list[str]):
         """
         Running this command removes the given packages from the given chroot.
         """
-        return ["arch-nspawn", chroot_dir, "pacman", "-Rsu", "--noconfirm"
+        return ["arch-nspawn", chroot_dir, "pacman", "-Rsu", "--noconfirm"] + packages
-                ] + packages
 
-    def make_chroot_pkg(self, chroot_wd_dir: str, user: str,
+    def make_chroot_pkg(
-                        pkgfiles_to_install: list[str]) -> list[str]:
+        self, chroot_wd_dir: str, user: str, pkgfiles_to_install: list[str]
+    ) -> list[str]:
         """
         Running this command creates a package file using the given chroot.
         The package is created as the user and the pkg_files_to_install are installed
         in the chroot before the package is created.
         """
-        makechrootpkg_cmd = [
+        makechrootpkg_cmd = ["makechrootpkg", "-c", "-r", chroot_wd_dir, "-U", user]
-            "makechrootpkg", "-c", "-r", chroot_wd_dir, "-U", user
-        ]
 
         for pkgfile in pkgfiles_to_install:
             makechrootpkg_cmd += ["-I", pkgfile]
@@ -199,6 +219,16 @@ valid_pkgexts: list[str] = [
     ".pkg.tar.Z",
 ]
 
+pacman_output_keywords: list[str] = [
+    "pacsave",
+    "pacnew",
+    # These cause too many false positives IMO
+    # "warning",
+    # "error",
+    # "note",
+]
+print_pacman_output_highlights: bool = True
+
 makepkg_user: str = "nobody"
 build_dir: str = "/tmp/decman/build"
 pkg_cache_dir: str = "/var/cache/decman"

src/decman/lib/__init__.py

@@ -2,16 +2,18 @@
 Library module for decman.
 """
 
-import pwd
-import shutil
-import subprocess
 import json
 import os
-import typing
+import shutil
+import subprocess
+import sys
+import threading
 import time
+import typing
+
+import decman
 import decman.config as conf
 import decman.error as err
-import decman
 
 _DECMAN_MSG_TAG = "[\033[1;35mDECMAN\033[m]"
 _RED_PREFIX = "\033[91m"
@@ -23,11 +25,15 @@ _RESET_SUFFIX = "\033[m"
 _SPACING = "    "
 _CONTINUATION_PREFIX = f"{_DECMAN_MSG_TAG}{_SPACING} "
 
+INFO = 1
+SUMMARY = 2
 
-def print_continuation(msg: str):
+
+def print_continuation(msg: str, level: int = SUMMARY):
     """
     Prints a message without a prefix.
     """
+    if level == SUMMARY or conf.debug_output or not conf.quiet_output:
         print(f"{_CONTINUATION_PREFIX}{msg}")
 
 
@@ -55,11 +61,14 @@ def print_summary(msg: str):
     print(f"{_DECMAN_MSG_TAG} {_CYAN_PREFIX}SUMMARY{_RESET_SUFFIX}: {msg}")
 
 
-def print_list_summary(msg: str,
+def print_list(
+    msg: str,
     l: list[str],
     elements_per_line: typing.Optional[int] = None,
     max_line_width: typing.Optional[int] = None,
-                       limit_to_term_size: bool = True):
+    limit_to_term_size: bool = True,
+    level: int = SUMMARY,
+):
     """
     Prints a summary message to the user along with a list of elements.
 
@@ -69,8 +78,12 @@ def print_list_summary(msg: str,
         return
 
     l = l.copy()
+    if level == SUMMARY:
         print_summary(msg)
-    print_continuation("")
+    elif level == INFO:
+        print_info(msg)
+
+    print_continuation("", level=level)
 
     if elements_per_line is None:
         elements_per_line = len(l)
@@ -79,8 +92,11 @@ def print_list_summary(msg: str,
         max_line_width = 2**32  # Big enough to basically be unlimited
 
     if limit_to_term_size:
-        max_line_width = shutil.get_terminal_size().columns - len(
+        max_line_width = (
-            _SPACING) - len(_CONTINUATION_PREFIX)
+            shutil.get_terminal_size().columns
+            - len(_SPACING)
+            - len(_CONTINUATION_PREFIX)
+        )
 
     lines = [f"{l.pop(0)}"]
     index = 0
@@ -100,9 +116,9 @@ def print_list_summary(msg: str,
             elements_in_current_line = 1
 
     for line in lines:
-        print_continuation(line)
+        print_continuation(line, level=level)
 
-    print_continuation("")
+    print_continuation("", level=level)
 
 
 def print_info(msg: str):
@@ -121,10 +137,9 @@ def print_debug(msg: str):
         print(f"{_DECMAN_MSG_TAG} {_GRAY_PREFIX}DEBUG{_RESET_SUFFIX}: {msg}")
 
 
-def prompt_number(msg: str,
+def prompt_number(
-                  min_num: int,
+    msg: str, min_num: int, max_num: int, default: typing.Optional[int] = None
-                  max_num: int,
+) -> int:
-                  default: typing.Optional[int] = None) -> int:
     """
     Prompts the user for a integer.
     """
@@ -253,24 +268,31 @@ class Store:
         if latest_path is None:
             return None
 
-        assert latest_version is not None, "If latest_path is set, then latest_version is set."
+        assert latest_version is not None, (
+            "If latest_path is set, then latest_version is set."
+        )
         return (latest_version, latest_path)
 
-    def add_package_to_cache(self, package: str, version: str,
+    def add_package_to_cache(self, package: str, version: str, path_to_built_pkg: str):
-                             path_to_built_pkg: str):
         """
         Adds a built package to the package file cache. Tries to remove excess cached packages.
         """
         new_entry = (version, path_to_built_pkg, int(time.time()))
         entries = self._package_file_cache.get(package, [])
+        for _, already_cached_path, __ in entries:
+            if already_cached_path == path_to_built_pkg:
+                print_debug(
+                    f"Trying to cache {package} version {version}, but the version is already cached: {already_cached_path}"
+                )
+                return
         entries.append(new_entry)
         self._package_file_cache[package] = entries
         self._clean_pkg_cache(package)
 
     def _clean_pkg_cache(self, package: str):
-        oldest_version = None
         oldest_path = None
         oldest_timestamp = None
+        index_of_oldest = None
 
         entries = self._package_file_cache[package]
         print_debug(f"Package cache has {len(entries)} entries.")
@@ -279,20 +301,19 @@ class Store:
             print_debug("Old files will not be removed.")
             return
 
-        for entry in entries:
+        for index, entry in enumerate(entries):
-            version, path, timestamp = entry
+            _, path, timestamp = entry
             if oldest_timestamp is None or oldest_timestamp > timestamp:
-                oldest_version = version
                 oldest_timestamp = timestamp
                 oldest_path = path
+                index_of_oldest = index
 
         print_debug(f"Oldest cached file for {package} is '{oldest_path}'.")
         if oldest_path is None:
             return
-        assert oldest_version is not None
+        assert index_of_oldest is not None
-        assert oldest_timestamp is not None
 
-        entries.remove((oldest_version, oldest_path, oldest_timestamp))
+        entries.pop(index_of_oldest)
         if os.path.exists(oldest_path):
             print_debug(f"Removing '{oldest_path}' from the package cache.")
             try:
@@ -317,14 +338,13 @@ class Store:
 
         d = {
             "source_file": self.source_file,
-            "allow_running_source_without_prompt":
+            "allow_running_source_without_prompt": self.allow_running_source_without_prompt,
-            self.allow_running_source_without_prompt,
             "enabled_systemd_units": self.enabled_systemd_units,
             "enabled_user_systemd_units": self._enabled_user_systemd_units,
             "enabled_modules": self.enabled_modules,
             "created_files": self.created_files,
             "package_file_cache": self._package_file_cache,
-            "pkgbuild_git_commits": self.pkgbuild_latest_reviewed_commits
+            "pkgbuild_git_commits": self.pkgbuild_latest_reviewed_commits,
         }
 
         try:
@@ -355,7 +375,8 @@ class Store:
 
                 store.source_file = d.get("source_file", None)
                 store.allow_running_source_without_prompt = d.get(
-                    "allow_running_source_without_prompt", False)
+                    "allow_running_source_without_prompt", False
+                )
                 store.enabled_systemd_units = d.get(
                     "enabled_systemd_units",
                     [],
@@ -375,12 +396,10 @@ class Store:
             return store
         except json.JSONDecodeError as e:
             print_error(f"{e}")
-            raise err.UserFacingError(
+            raise err.UserFacingError("Failed to parse decman store json.") from e
-                "Failed to parse decman store json.") from e
         except OSError as e:
             print_error(f"{e}")
-            raise err.UserFacingError(
+            raise err.UserFacingError("Failed to read saved decman store.") from e
-                "Failed to read saved decman store.") from e
 
 
 class Source:
@@ -440,40 +459,53 @@ class Source:
         """
         for module in self.modules:
             if module.enabled and module.version != store.enabled_modules.get(
-                    module.name, module.version):
+                module.name, module.version
+            ):
                 module.after_version_change()
             elif module.enabled and module.name not in store.enabled_modules:
                 module.after_version_change()
 
-    def create_all_files(self) -> list[str]:
+    def create_all_files(self, only_print: bool) -> list[str]:
         """
         Creates all files and returns them. The files created are based on the specified files,
         directories and modules.
         """
         created_files = []
 
-        def install_files(files: dict[str, decman.File],
+        def install_files(
-                          variables: typing.Optional[dict[str, str]] = None):
+            files: dict[str, decman.File],
+            variables: typing.Optional[dict[str, str]] = None,
+        ):
             for target, file in files.items():
                 created_files.append(target)
+
+                if only_print:
+                    continue
+
                 try:
-                    file.copy_to(target, variables)
                     print_debug(f"Installing file to {target}.")
+                    file.copy_to(target, variables)
                 except OSError as e:
                     print_error(f"{e}")
                     raise err.UserFacingError(
-                        f"Failed to install file to {target}.") from e
+                        f"Failed to install file to {target}."
+                    ) from e
 
-        def install_dirs(dirs: dict[str, decman.Directory],
+        def install_dirs(
-                         variables: typing.Optional[dict[str, str]] = None):
+            dirs: dict[str, decman.Directory],
+            variables: typing.Optional[dict[str, str]] = None,
+        ):
             for target, directory in dirs.items():
                 try:
                     print_debug(f"Installing directory to {target}.")
-                    created_files.extend(directory.copy_to(target, variables))
+                    created_files.extend(
+                        directory.copy_to(target, variables, only_print)
+                    )
                 except OSError as e:
                     print_error(f"{e}")
                     raise err.UserFacingError(
-                        f"Failed to install directory to {target}.") from e
+                        f"Failed to install directory to {target}."
+                    ) from e
 
         install_files(self.files)
         install_dirs(self.directories)
@@ -511,8 +543,7 @@ class Source:
 
         return all_dirs
 
-    def files_to_remove(self, store: Store,
+    def files_to_remove(self, store: Store, created_files: list[str]) -> list[str]:
-                        created_files: list[str]) -> list[str]:
         """
         Returns all files that should be removed.
         """
@@ -567,8 +598,7 @@ class Source:
                 result[user] = entry
         return result
 
-    def packages_to_remove(
+    def packages_to_remove(self, currently_installed_packages: list[str]) -> list[str]:
-            self, currently_installed_packages: list[str]) -> list[str]:
         """
         Returns all packages that should be removed. This includes pacman, aur and user packages.
         """
@@ -581,7 +611,8 @@ class Source:
         return result
 
     def pacman_packages_to_install(
-            self, currently_installed_packages: list[str]) -> list[str]:
+        self, currently_installed_packages: list[str]
+    ) -> list[str]:
         """
         Returns all pacman packages that should be installed.
         """
@@ -594,7 +625,8 @@ class Source:
         return result
 
     def foreign_packages_to_install(
-            self, currently_installed_packages: list[str]) -> list[str]:
+        self, currently_installed_packages: list[str]
+    ) -> list[str]:
         """
         Returns all aur and user packages that should be installed.
         """
@@ -660,11 +692,13 @@ class Source:
         return result
 
     def _all_user_units(self) -> dict[str, set[str]]:
-        result = {}
+        result = self.systemd_user_units
-        result.update(self.systemd_user_units)
+        for module in [m for m in self.modules if m.enabled]:
-        for module in self.modules:
+            module_user_units: dict[str, list[str]] = module.systemd_user_units()
-            if module.enabled:
+            for user in module_user_units.keys():
-                result.update(module.systemd_user_units())
+                if user not in result:
+                    result[user] = set()
+                result[user].update(module_user_units[user])
         return result
 
 
@@ -682,11 +716,16 @@ class Pacman:
         """
 
         try:
-            packages = subprocess.run(
+            packages = (
+                subprocess.run(
                     conf.commands.list_pkgs(),
                     check=True,
                     stdout=subprocess.PIPE,
-            ).stdout.decode().strip().split('\n')
+                )
+                .stdout.decode()
+                .strip()
+                .split("\n")
+            )
             return packages
         except subprocess.CalledProcessError as error:
             raise err.UserFacingError(
@@ -700,9 +739,12 @@ class Pacman:
         if dep in self._installable:
             return self._installable[dep]
 
-        result = subprocess.run(conf.commands.is_installable(dep),
+        result = (
-                                check=False,
+            subprocess.run(
-                                capture_output=True).returncode == 0
+                conf.commands.is_installable(dep), check=False, capture_output=True
+            ).returncode
+            == 0
+        )
         self._installable[dep] = result
         return result
 
@@ -712,18 +754,23 @@ class Pacman:
         basically AUR packages.
         """
         try:
-            output = subprocess.run(
+            output = (
+                subprocess.run(
                     conf.commands.list_foreign_pkgs_versioned(),
                     check=True,
-                stdout=subprocess.PIPE).stdout.decode().strip().split('\n')
+                    stdout=subprocess.PIPE,
+                )
+                .stdout.decode()
+                .strip()
+                .split("\n")
+            )
         except subprocess.CalledProcessError as error:
             raise err.UserFacingError(
                 f"Failed to get foreign packages using '{error.cmd}'. Output: {error.stdout}."
             ) from error
 
         try:
-            return [(line.split(" ")[0], line.split(" ")[1])
+            return [(line.split(" ")[0], line.split(" ")[1]) for line in output]
-                    for line in output]
         except IndexError as error:
             raise err.UserFacingError(
                 f"Failed to parse foreign packages from pacman output. Output: {output}"
@@ -736,11 +783,26 @@ class Pacman:
         if not packages:
             return
 
+        returncode, output = echo_and_capture_command(
+            conf.commands.install_pkgs(packages)
+        )
+        if returncode != 0:
+            raise err.UserFacingError(
+                f"Failed to install packages using pacman. Process exited with code {returncode}."
+            )
+        if conf.print_pacman_output_highlights:
+            print_highlighted_pacman_messages(output)
+
         try:
-            subprocess.run(conf.commands.install_pkgs(packages), check=True)
+            subprocess.run(
+                conf.commands.set_as_explicitly_installed(packages),
+                check=True,
+                capture_output=conf.suppress_command_output,
+            )
         except subprocess.CalledProcessError as error:
             raise err.UserFacingError(
-                "Failed to install packages using pacman.") from error
+                "Failed to set packages as explicitly installed using pacman."
+            ) from error
 
     def install_dependencies(self, deps: list[str]):
         """
@@ -749,12 +811,13 @@ class Pacman:
         if not deps:
             return
 
-        try:
+        returncode, output = echo_and_capture_command(conf.commands.install_deps(deps))
-            subprocess.run(conf.commands.install_deps(deps), check=True)
+        if returncode != 0:
-        except subprocess.CalledProcessError as error:
             raise err.UserFacingError(
-                "Failed to install packages as dependencies using pacman."
+                f"Failed to install packages as dependencies using pacman. Process exited with code {returncode}."
-            ) from error
+            )
+        if conf.print_pacman_output_highlights:
+            print_highlighted_pacman_messages(output)
 
     def install_files(self, files: list[str], as_explicit: list[str]):
         """
@@ -764,30 +827,42 @@ class Pacman:
         if not files:
             return
 
-        try:
+        returncode, output = echo_and_capture_command(
-            subprocess.run(conf.commands.install_files(files), check=True)
+            conf.commands.install_files(files)
+        )
+        if returncode != 0:
+            raise err.UserFacingError(
+                f"Failed to install package files using pacman. Process exited with code {returncode}."
+            )
+        if conf.print_pacman_output_highlights:
+            print_highlighted_pacman_messages(output)
 
+        try:
             if as_explicit:
                 subprocess.run(
                     conf.commands.set_as_explicitly_installed(as_explicit),
                     check=True,
-                    capture_output=conf.suppress_command_output)
+                    capture_output=conf.suppress_command_output,
+                )
         except subprocess.CalledProcessError as error:
             if conf.suppress_command_output:
                 print_error("Output:")
                 print_continuation(error.output)
             raise err.UserFacingError(
-                "Failed to install package files using pacman.") from error
+                "Failed to set packages as explicitly installed using pacman."
+            ) from error
 
     def upgrade(self):
         """
         Upgrades all packages.
         """
-        try:
+        returncode, output = echo_and_capture_command(conf.commands.upgrade())
-            subprocess.run(conf.commands.upgrade(), check=True)
+        if returncode != 0:
-        except subprocess.CalledProcessError as error:
             raise err.UserFacingError(
-                "Failed to upgrade packages using pacman.") from error
+                f"Failed to upgrade packages using pacman. Process exited with code {returncode}."
+            )
+        if conf.print_pacman_output_highlights:
+            print_highlighted_pacman_messages(output)
 
     def remove(self, packages: list[str]):
         """
@@ -795,11 +870,92 @@ class Pacman:
         """
         if not packages:
             return
-        try:
+
-            subprocess.run(conf.commands.remove(packages), check=True)
+        returncode, output = echo_and_capture_command(conf.commands.remove(packages))
-        except subprocess.CalledProcessError as error:
+        if returncode != 0:
             raise err.UserFacingError(
-                "Failed to remove packages using pacman.") from error
+                f"Failed to remove packages using pacman. Process exited with code {returncode}."
+            )
+        if conf.print_pacman_output_highlights:
+            print_highlighted_pacman_messages(output)
+
+
+def print_highlighted_pacman_messages(output: str):
+    """
+    Prints lines that contain pacman output keywords.
+    """
+    print_summary("Pacman output highlights:")
+    lines = output.split("\n")
+    for index, line in enumerate(lines):
+        for keyword in conf.pacman_output_keywords:
+            if keyword.lower() in line.lower():
+                print_summary(f"lines: {index}-{index + 2}")
+                if index >= 1:
+                    print_continuation(lines[index - 1])
+                print_continuation(line)
+                if index + 1 < len(lines):
+                    print_continuation(lines[index + 1])
+                print_continuation("")
+
+                # Break, as to not print the same line again if it contains multiple keywords
+                break
+
+
+def echo_and_capture_command(program: list[str]) -> tuple[int, str]:
+    """
+    Runs the given CLI program and arguments.
+
+    Returns a tuple containing the return code of the program as well as all output of the program.
+    """
+    with subprocess.Popen(
+        program, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.STDOUT
+    ) as process:
+        os.set_blocking(process.stdout.fileno(), False)
+
+        output_thread = _OutputCapturingThread(process.stdout)
+        output_thread.start()
+
+        os.set_blocking(sys.stdin.fileno(), False)
+
+        # Capture stdin and forward it to the process in a non-blocking manner
+        while process.poll() is None:
+            inp = sys.stdin.readline()
+
+            if inp:
+                process.stdin.write(inp.encode())
+                process.stdin.flush()
+
+            time.sleep(0.1)
+
+        os.set_blocking(sys.stdin.fileno(), True)
+
+        output_thread.done = True
+        output_thread.join()
+
+        # Capture any output that may not have been yet captured
+        output = output_thread.output
+        missing_output = process.stdout.read()
+        if missing_output:
+            output += missing_output.decode()
+
+        return (process.returncode, output)
+
+
+class _OutputCapturingThread(threading.Thread):
+    def __init__(self, stream):
+        super().__init__()
+        self._stream = stream
+        self.output = ""
+        self.done = False
+
+    def run(self):
+        while not self.done and not self._stream.closed:
+            output = self._stream.read(1000)
+            if output:
+                output = output.decode()
+                self.output += output
+                print(output, end="", flush=True)
+            time.sleep(0.1)
 
 
 class Systemd:
@@ -818,10 +974,15 @@ class Systemd:
             return
 
         try:
-            subprocess.run(conf.commands.enable_units(units), check=True)
+            subprocess.run(
+                conf.commands.enable_units(units),
+                check=True,
+                capture_output=conf.suppress_command_output,
+            )
         except subprocess.CalledProcessError as error:
             raise err.UserFacingError(
-                f"Failed to enable systemd units: {units}") from error
+                f"Failed to enable systemd units: {units}"
+            ) from error
         self.state.enabled_systemd_units += units
 
     def disable_units(self, units: list[str]):
@@ -832,10 +993,15 @@ class Systemd:
             return
 
         try:
-            subprocess.run(conf.commands.disable_units(units), check=True)
+            subprocess.run(
+                conf.commands.disable_units(units),
+                check=True,
+                capture_output=conf.suppress_command_output,
+            )
         except subprocess.CalledProcessError as error:
             raise err.UserFacingError(
-                f"Failed to disable systemd units: {units}") from error
+                f"Failed to disable systemd units: {units}"
+            ) from error
         for unit in units:
             try:
                 self.state.enabled_systemd_units.remove(unit)
@@ -850,19 +1016,16 @@ class Systemd:
             return
 
         try:
-            uid = pwd.getpwnam(user).pw_uid
+            subprocess.run(
-            gid = pwd.getpwnam(user).pw_gid
+                conf.commands.enable_user_units(units, user),
-
+                check=True,
-            with subprocess.Popen(conf.commands.enable_user_units(units),
+                capture_output=conf.suppress_command_output,
-                                  group=gid,
+            )
-                                  user=uid) as process:
+        except subprocess.CalledProcessError as error:
-                if process.wait() != 0:
             raise err.UserFacingError(
-                        f"Failed to enable systemd units: {units} for {user}.")
+                f"Failed to enable systemd units: {units} for {user}."
-        except KeyError as error:
-            raise err.UserFacingError(
-                f"Failed to enable systemd units because user {user} doesn't exist."
             ) from error
+
         for unit in units:
             self.state.add_enabled_user_systemd_unit(user, unit)
 
@@ -874,19 +1037,14 @@ class Systemd:
             return
 
         try:
-            uid = pwd.getpwnam(user).pw_uid
+            subprocess.run(
-            gid = pwd.getpwnam(user).pw_gid
+                conf.commands.disable_user_units(units, user),
-
+                check=True,
-            with subprocess.Popen(conf.commands.disable_user_units(units),
+                capture_output=conf.suppress_command_output,
-                                  group=gid,
+            )
-                                  user=uid) as process:
+        except subprocess.CalledProcessError as error:
-                if process.wait() != 0:
             raise err.UserFacingError(
                 f"Failed to disable systemd units: {units} for {user}."
-                    )
-        except KeyError as error:
-            raise err.UserFacingError(
-                f"Failed to disable systemd units because user {user} doesn't exist."
             ) from error
 
         for unit in units:

src/decman/lib/fpm.py

@@ -11,18 +11,18 @@ Terminology:
 - all dependencies: normal dependencies and build dependencies combined
 """
 
-import shutil
-import subprocess
 import os
 import re
+import shutil
+import subprocess
 import typing
 
 import requests
 
 import decman
 import decman.config as conf
-import decman.lib as l
 import decman.error as err
+import decman.lib as l
 
 
 def strip_dependency(dep: str) -> str:
@@ -58,10 +58,18 @@ class PackageInfo:
     In case of AUR packages, these are fetched from AUR RPC.
     """
 
-    def __init__(self, pkgname: str, pkgbase: str, version: str,
+    def __init__(
-                 provides: list[str], dependencies: list[str],
+        self,
-                 make_dependencies: list[str], check_dependencies: list[str],
+        pkgname: str,
-                 git_url: str, pacman: l.Pacman):
+        pkgbase: str,
+        version: str,
+        provides: list[str],
+        dependencies: list[str],
+        make_dependencies: list[str],
+        check_dependencies: list[str],
+        git_url: str,
+        pacman: l.Pacman,
+    ):
         self.pkgname = pkgname
         self.pkgbase = pkgbase
         self.version = version
@@ -79,22 +87,23 @@ class PackageInfo:
             if pacman.is_installable(dep):
                 self.pacman_dependencies.append(dep)
             else:
-                self.foreign_dependencies_stripped.append(
+                self.foreign_dependencies_stripped.append(strip_dependency(dep))
-                    strip_dependency(dep))
 
         for make_dep in make_dependencies:
             if pacman.is_installable(make_dep):
                 self.pacman_make_dependencies.append(make_dep)
             else:
                 self.foreign_make_dependencies_stripped.append(
-                    strip_dependency(make_dep))
+                    strip_dependency(make_dep)
+                )
 
         for check_dep in check_dependencies:
             if pacman.is_installable(check_dep):
                 self.pacman_check_dependencies.append(check_dep)
             else:
                 self.foreign_check_dependencies_stripped.append(
-                    strip_dependency(check_dep))
+                    strip_dependency(check_dep)
+                )
 
     def pkg_file_prefix(self) -> str:
         """
@@ -103,8 +112,9 @@ class PackageInfo:
         return f"{self.pkgname}-{self.version}"
 
     @staticmethod
-    def from_user_package(user_package: decman.UserPackage,
+    def from_user_package(
-                          pacman: l.Pacman) -> "PackageInfo":
+        user_package: decman.UserPackage, pacman: l.Pacman
+    ) -> "PackageInfo":
         """
         Converts a UserPackage to PackageInfo
         """
@@ -132,8 +142,11 @@ class ForeignPackage:
 
     def __eq__(self, value: object, /) -> bool:
         if isinstance(value, self.__class__):
-            return self.name == value.name \
+            return (
-                   and self._all_recursive_foreign_deps == value._all_recursive_foreign_deps
+                self.name == value.name
+                and self._all_recursive_foreign_deps
+                == value._all_recursive_foreign_deps
+            )
         return False
 
     def __hash__(self) -> int:
@@ -145,8 +158,7 @@ class ForeignPackage:
     def __str__(self) -> str:
         return f"{self.name}"
 
-    def add_foreign_dependency_packages(self,
+    def add_foreign_dependency_packages(self, package_names: typing.Iterable[str]):
-                                        package_names: typing.Iterable[str]):
         """
         Adds dependencies to the package.
         """
@@ -174,8 +186,7 @@ class DepNode:
         Returns True if the given package name is in the parents of this DepNode.
         """
         for name, parent in self.parents.items():
-            if name == pkgname or parent.is_pkgname_in_parents_recursive(
+            if name == pkgname or parent.is_pkgname_in_parents_recursive(pkgname):
-                    pkgname):
                 return True
         return False
 
@@ -189,15 +200,15 @@ class DepGraph:
         self.package_nodes: dict[str, DepNode] = {}
         self._childless_node_names = set()
 
-    def add_requirement(self, child_pkgname: str,
+    def add_requirement(self, child_pkgname: str, parent_pkgname: typing.Optional[str]):
-                        parent_pkgname: typing.Optional[str]):
         """
         Adds a connection between two packages, creating the child package if it doesn't exist.
 
         The parent is the package that requires the child package.
         """
         child_node = self.package_nodes.get(
-            child_pkgname, DepNode(ForeignPackage(child_pkgname)))
+            child_pkgname, DepNode(ForeignPackage(child_pkgname))
+        )
         self.package_nodes[child_pkgname] = child_node
 
         if len(child_node.children) == 0:
@@ -212,7 +223,8 @@ class DepGraph:
             raise err.UserFacingError(
                 f"Foreign package dependency cycle detected involving '{child_pkgname}' \
 and '{parent_pkgname}'. Foreign package dependencies are also required \
-during package building and therefore dependency cycles cannot be handled.")
+during package building and therefore dependency cycles cannot be handled."
+            )
 
         parent_node.children[child_pkgname] = child_node
         child_node.parents[parent_pkgname] = parent_node
@@ -230,8 +242,7 @@ during package building and therefore dependency cycles cannot be handled.")
             childless_node = self.package_nodes[childless_node_name]
 
             for parent in childless_node.parents.values():
-                new_deps = childless_node.pkg.get_all_recursive_foreign_dep_pkgs(
+                new_deps = childless_node.pkg.get_all_recursive_foreign_dep_pkgs()
-                )
                 new_deps.add(childless_node.pkg.name)
                 parent.pkg.add_foreign_dependency_packages(new_deps)
                 del parent.children[childless_node_name]
@@ -254,6 +265,7 @@ class ExtendedPackageSearch:
         self._pacman = pacman
         self._package_info_cache: dict[str, PackageInfo] = {}
         self._dep_provider_cache: dict[str, PackageInfo] = {}
+        self._known_providers_cache: dict[str, list[str]] = {}
         self._user_packages: list[PackageInfo] = []
 
     def add_user_pkg(self, user_pkg: PackageInfo):
@@ -261,6 +273,15 @@ class ExtendedPackageSearch:
         Adds the given package to user packages.
         """
         self._user_packages.append(user_pkg)
+        self._cache_pkg(user_pkg)
+
+    def _cache_pkg(self, pkg: PackageInfo):
+        for provided_pkg in pkg.provides:
+            self._known_providers_cache[provided_pkg] = self._known_providers_cache.get(
+                provided_pkg, []
+            )
+            self._known_providers_cache[provided_pkg].append(pkg.pkgname)
+        self._package_info_cache[pkg.pkgname] = pkg
 
     def try_caching_packages(self, packages: list[str]):
         """
@@ -270,8 +291,7 @@ class ExtendedPackageSearch:
         times, because then those methods don't have to make new AUR RPC requests.
         """
 
-        packages = list(
+        packages = list(filter(lambda p: p not in self._package_info_cache, packages))
-            filter(lambda p: p not in self._package_info_cache, packages))
 
         if len(packages) == 0:
             return
@@ -281,8 +301,7 @@ class ExtendedPackageSearch:
         max_pkgs_per_request = 200
 
         while packages:
-            to_request = map(lambda p: f"arg[]={p}",
+            to_request = map(lambda p: f"arg[]={p}", packages[:max_pkgs_per_request])
-                             packages[:max_pkgs_per_request])
             packages = packages[max_pkgs_per_request:]
 
             url = f"https://aur.archlinux.org/rpc/v5/info?{'&'.join(to_request)}"
@@ -293,8 +312,7 @@ class ExtendedPackageSearch:
                 d = request.json()
 
                 if d["type"] == "error":
-                    raise err.UserFacingError(
+                    raise err.UserFacingError(f"AUR RPC returned error: {d['error']}")
-                        f"AUR RPC returned error: {d['error']}")
 
                 for result in d["results"]:
                     pkgname = result["Name"]
@@ -304,9 +322,8 @@ class ExtendedPackageSearch:
 
                     for user_package in self._user_packages:
                         if user_package.pkgname == pkgname:
-                            l.print_debug(
+                            l.print_debug(f"'{pkgname}' found in user packages.")
-                                f"'{pkgname}' found in user packages.")
+                            self._cache_pkg(user_package)
-                            self._package_info_cache[pkgname] = user_package
                             break
                     else:  # if not in user_packages then:
                         info = PackageInfo(
@@ -317,10 +334,10 @@ class ExtendedPackageSearch:
                             make_dependencies=result.get("MakeDepends", []),
                             check_dependencies=result.get("CheckDepends", []),
                             provides=result.get("Provides", []),
-                            git_url=
+                            git_url=f"https://aur.archlinux.org/{result['PackageBase']}.git",
-                            f"https://aur.archlinux.org/{result['PackageBase']}.git",
+                            pacman=self._pacman,
-                            pacman=self._pacman)
+                        )
-                        self._package_info_cache[pkgname] = info
+                        self._cache_pkg(info)
 
                 l.print_debug("Request completed.")
             except (requests.RequestException, KeyError) as e:
@@ -345,7 +362,7 @@ class ExtendedPackageSearch:
         for user_package in self._user_packages:
             if user_package.pkgname == package:
                 l.print_debug(f"'{package}' found in user packages.")
-                self._package_info_cache[package] = user_package
+                self._cache_pkg(user_package)
                 return user_package
 
         url = f"https://aur.archlinux.org/rpc/v5/info/{package}"
@@ -355,8 +372,7 @@ class ExtendedPackageSearch:
             d = request.json()
 
             if d["type"] == "error":
-                raise err.UserFacingError(
+                raise err.UserFacingError(f"AUR RPC returned error: {d['error']}")
-                    f"AUR RPC returned error: {d['error']}")
 
             if d["resultcount"] == 0:
                 l.print_debug(f"'{package}' not found.")
@@ -373,11 +389,11 @@ class ExtendedPackageSearch:
                 make_dependencies=result.get("MakeDepends", []),
                 check_dependencies=result.get("CheckDepends", []),
                 provides=result.get("Provides", []),
-                git_url=
+                git_url=f"https://aur.archlinux.org/{result['PackageBase']}.git",
-                f"https://aur.archlinux.org/{result['PackageBase']}.git",
+                pacman=self._pacman,
-                pacman=self._pacman)
+            )
 
-            self._package_info_cache[package] = info
+            self._cache_pkg(info)
 
             return info
         except (requests.RequestException, KeyError) as e:
@@ -386,8 +402,7 @@ class ExtendedPackageSearch:
                 f"Failed to fetch package information for {package} from AUR RPC."
             ) from e
 
-    def find_provider(
+    def find_provider(self, stripped_dependency: str) -> typing.Optional[PackageInfo]:
-            self, stripped_dependency: str) -> typing.Optional[PackageInfo]:
         """
         Finds a provider for a dependency.
 
@@ -410,25 +425,31 @@ class ExtendedPackageSearch:
 
         l.print_debug("No exact name matches found. Finding providers.")
 
-        user_pkg_results = []
+        known_pkg_results = self._known_providers_cache.get(stripped_dependency, [])
         for user_package in self._user_packages:
-            if stripped_dependency in user_package.provides:
+            if (
-                user_pkg_results.append(user_package.pkgname)
+                stripped_dependency in user_package.provides
+                and stripped_dependency not in known_pkg_results
+            ):
+                known_pkg_results.append(user_package.pkgname)
 
-        if len(user_pkg_results) == 1:
+        if len(known_pkg_results) == 1:
-            pkg = self.get_package_info(user_pkg_results[0])
+            pkg = self.get_package_info(known_pkg_results[0])
             assert pkg is not None
             l.print_debug(
-                f"Single provider for '{stripped_dependency}' found in user packages: '{pkg}'."
+                f"Single provider for '{stripped_dependency}' found in known packages: '{pkg}'."
             )
             self._dep_provider_cache[stripped_dependency] = pkg
             return pkg
 
-        if len(user_pkg_results) > 1:
+        if len(known_pkg_results) > 1:
-            return self._choose_provider(stripped_dependency, user_pkg_results,
+            return self._choose_provider(
-                                         "user packages")
+                stripped_dependency, known_pkg_results, "user packages"
+            )
 
-        url = f"https://aur.archlinux.org/rpc/v5/search/{stripped_dependency}?by=provides"
+        url = (
+            f"https://aur.archlinux.org/rpc/v5/search/{stripped_dependency}?by=provides"
+        )
         l.print_debug(
             f"Requesting providers for '{stripped_dependency}' from AUR. URL = {url}"
         )
@@ -437,8 +458,7 @@ class ExtendedPackageSearch:
             d = request.json()
 
             if d["type"] == "error":
-                raise err.UserFacingError(
+                raise err.UserFacingError(f"AUR RPC returned error: {d['error']}")
-                    f"AUR RPC returned error: {d['error']}")
 
             if d["resultcount"] == 0:
                 l.print_debug(f"'{stripped_dependency}' not found.")
@@ -461,8 +481,9 @@ class ExtendedPackageSearch:
                 f"Failed to search for {stripped_dependency} from AUR RPC."
             ) from e
 
-    def _choose_provider(self, dep: str, possible_providers: list[str],
+    def _choose_provider(
-                         where: str) -> typing.Optional[PackageInfo]:
+        self, dep: str, possible_providers: list[str], where: str
+    ) -> typing.Optional[PackageInfo]:
         min_selection = 1
         max_selection = len(possible_providers)
         l.print_summary(
@@ -472,13 +493,14 @@ class ExtendedPackageSearch:
         providers = "Providers: "
         for index, name in enumerate(possible_providers):
             providers += f"{index + 1}:{name} "
-        l.print_info(providers)
+        l.print_summary(providers)
 
         selection = l.prompt_number(
             f"Select a provider [{min_selection}-{max_selection}] (default: {min_selection}): ",
             min_selection,
             max_selection,
-            default=min_selection)
+            default=min_selection,
+        )
 
         info = self.get_package_info(possible_providers[selection - 1])
         if info is not None:
@@ -541,31 +563,30 @@ class ForeignPackageManager:
     Class for dealing with foreign packages.
     """
 
-    def __init__(self, store: l.Store, pacman: l.Pacman,
+    def __init__(self, store: l.Store, pacman: l.Pacman, search: ExtendedPackageSearch):
-                 search: ExtendedPackageSearch):
         self._store = store
         self._pacman = pacman
         self._search = search
 
-    def upgrade(self,
+    def upgrade(
+        self,
         upgrade_devel: bool = False,
         force: bool = False,
-                ignored_pkgs: typing.Optional[set[str]] = None):
+        ignored_pkgs: typing.Optional[set[str]] = None,
+    ):
         """
         Upgrades all foreign packages.
         """
         if ignored_pkgs is None:
             ignored_pkgs = set()
 
-        l.print_summary("Determining packages to upgrade.")
+        l.print_summary("Determining foreign packages to upgrade.")
 
         all_foreign_pkgs = self._pacman.get_versioned_foreign_packages()
         all_explicit_pkgs = set(self._pacman.get_installed())
-        l.print_debug(
+        l.print_debug(f"Foreign packages to check for upgrades: {all_foreign_pkgs}")
-            f"Foreign packages to check for upgrades: {all_foreign_pkgs}")
 
-        self._search.try_caching_packages(
+        self._search.try_caching_packages(list(map(lambda p: p[0], all_foreign_pkgs)))
-            list(map(lambda p: p[0], all_foreign_pkgs)))
 
         as_explicit = []
         as_deps = []
@@ -579,8 +600,7 @@ class ForeignPackageManager:
                     f"Failed to find '{pkg}' from AUR or user provided packages."
                 )
 
-            if self.should_upgrade_package(pkg, ver, info.version,
+            if self.should_upgrade_package(pkg, ver, info.version, upgrade_devel):
-                                           upgrade_devel):
                 if pkg in all_explicit_pkgs:
                     as_explicit.append(pkg)
                 else:
@@ -592,10 +612,12 @@ class ForeignPackageManager:
 
         self.install(as_explicit, as_deps, force)
 
-    def install(self,
+    def install(
+        self,
         foreign_pkgs: list[str],
         foreign_dep_pkgs: typing.Optional[list[str]] = None,
-                force: bool = False):
+        force: bool = False,
+    ):
         """
         Installs the given foreign packages and their dependencies (both pacman/AUR).
         """
@@ -607,36 +629,44 @@ class ForeignPackageManager:
             return
 
         resolved_dependencies = self.resolve_dependencies(
-            foreign_pkgs, foreign_dep_pkgs)
+            foreign_pkgs, foreign_dep_pkgs
+        )
 
-        l.print_list_summary(
+        l.print_list(
             "The following foreign packages will be installed explicitly:",
-            list(resolved_dependencies.foreign_pkgs))
+            list(resolved_dependencies.foreign_pkgs),
+            level=l.SUMMARY,
+        )
 
-        l.print_list_summary(
+        l.print_list(
             "The following foreign packages will be installed as dependencies:",
-            list(resolved_dependencies.foreign_dep_pkgs))
+            list(resolved_dependencies.foreign_dep_pkgs),
+            level=l.SUMMARY,
+        )
 
-        l.print_list_summary(
+        l.print_list(
             "The following foreign packages will be built in order to install other packages. They will not be installed:",
-            list(resolved_dependencies.foreign_build_dep_pkgs))
+            list(resolved_dependencies.foreign_build_dep_pkgs),
+            level=l.SUMMARY,
+        )
 
         if not l.prompt_confirm("Proceed?", default=True):
             raise err.UserFacingError("Installing aborted.")
 
         l.print_summary("Installing foreign package dependencies from pacman.")
-        self._pacman.install_dependencies(
+        self._pacman.install_dependencies(list(resolved_dependencies.pacman_deps))
-            list(resolved_dependencies.pacman_deps))
 
         try:
-            with PackageBuilder(self._search, self._store,
+            with PackageBuilder(
-                                resolved_dependencies) as builder:
+                self._search, self._store, resolved_dependencies
+            ) as builder:
                 while resolved_dependencies.build_order:
                     to_build = resolved_dependencies.build_order.pop(0)
 
                     pkgbase = resolved_dependencies.get_pkgbase(to_build)
                     package_names = resolved_dependencies.get_pkgs_with_common_pkgbase(
-                        to_build)
+                        to_build
+                    )
 
                     packages = [
                         resolved_dependencies.packages[pkgname]
@@ -660,22 +690,23 @@ class ForeignPackageManager:
 
         if package_files_to_install or force:
             l.print_summary("Installing foreign packages.")
-            self._pacman.install_files(package_files_to_install,
+            self._pacman.install_files(
-                                       as_explicit=list(
+                package_files_to_install,
-                                           resolved_dependencies.foreign_pkgs))
+                as_explicit=list(resolved_dependencies.foreign_pkgs),
+            )
         else:
             l.print_summary("No packages to install.")
 
     def resolve_dependencies(
         self,
         foreign_pkgs: list[str],
-        foreign_dep_pkgs: typing.Optional[list[str]] = None
+        foreign_dep_pkgs: typing.Optional[list[str]] = None,
     ) -> ResolvedDependencies:
         """
         Resolves foreign dependencies of foreign packages.
         """
 
-        l.print_summary("Resolving foreign package dependencies.")
+        l.print_info("Resolving foreign package dependencies.")
         l.print_debug(f"Packages: {foreign_pkgs}")
 
         if foreign_dep_pkgs is None:
@@ -687,7 +718,7 @@ class ForeignPackageManager:
 
         graph = DepGraph()
 
-        for name in (foreign_pkgs + foreign_dep_pkgs):
+        for name in foreign_pkgs + foreign_dep_pkgs:
             graph.add_requirement(name, None)
 
         seen_packages = set(foreign_pkgs + foreign_dep_pkgs)
@@ -706,8 +737,7 @@ class ForeignPackageManager:
 
             add_to.add(dep_info.pkgname)
 
-            l.print_debug(
+            l.print_debug(f"Adding dependency {dep_info.pkgname} to package {pkgname}.")
-                f"Adding dependency {dep_info.pkgname} to package {pkgname}.")
             graph.add_requirement(dep_info.pkgname, pkgname)
             if dep_info.pkgname not in seen_packages:
                 to_process.append(dep_info.pkgname)
@@ -725,10 +755,14 @@ class ForeignPackageManager:
             result.pacman_deps.update(info.pacman_dependencies)
             result.add_pkgbase_info(pkgname, info.pkgbase)
 
-            build_deps = info.foreign_make_dependencies_stripped + info.foreign_check_dependencies_stripped
+            build_deps = (
+                info.foreign_make_dependencies_stripped
+                + info.foreign_check_dependencies_stripped
+            )
 
             self._search.try_caching_packages(
-                info.foreign_dependencies_stripped + build_deps)
+                info.foreign_dependencies_stripped + build_deps
+            )
 
             for depname in info.foreign_dependencies_stripped:
                 process_dep(pkgname, depname, result.foreign_dep_pkgs)
@@ -739,7 +773,7 @@ class ForeignPackageManager:
             total_processed += 1
             l.print_info(f"Progress: {total_processed}/{len(seen_packages)}.")
 
-        l.print_summary("Determining build order.")
+        l.print_info("Determining build order.")
 
         while True:
             to_add = graph.get_and_remove_outer_dep_pkgs()
@@ -755,26 +789,29 @@ class ForeignPackageManager:
 
         return result
 
-    def should_upgrade_package(self,
+    def should_upgrade_package(
+        self,
         package: str,
         installed_version: str,
         fetched_version: str,
-                               upgrade_devel=False) -> bool:
+        upgrade_devel=False,
+    ) -> bool:
         """
         Returns True if a package should be upgraded.
         """
 
         if upgrade_devel and is_devel(package):
-            l.print_debug(
+            l.print_debug(f"Package {package} is devel package. It should be upgraded.")
-                f"Package {package} is devel package. It should be upgraded.")
             return True
 
         try:
             result = int(
-                subprocess.run(conf.commands.compare_versions(
+                subprocess.run(
-                    installed_version, fetched_version),
+                    conf.commands.compare_versions(installed_version, fetched_version),
                     check=True,
-                               stdout=subprocess.PIPE).stdout.decode())
+                    stdout=subprocess.PIPE,
+                ).stdout.decode()
+            )
             should_upgrade = result < 0
             l.print_debug(
                 f"Installed version is: {installed_version}. Available version is {fetched_version}. Should upgrade: {should_upgrade}"
@@ -783,7 +820,8 @@ class ForeignPackageManager:
         except (ValueError, subprocess.CalledProcessError) as error:
             l.print_error(f"{error}")
             raise err.UserFacingError(
-                "Failed to compare versions using vercmp.") from error
+                "Failed to compare versions using vercmp."
+            ) from error
 
 
 class PackageBuilder:
@@ -793,8 +831,12 @@ class PackageBuilder:
 
     always_included_packages = ["base-devel", "git"]
 
-    def __init__(self, search: ExtendedPackageSearch, store: l.Store,
+    def __init__(
-                 resolved_deps: ResolvedDependencies):
+        self,
+        search: ExtendedPackageSearch,
+        store: l.Store,
+        resolved_deps: ResolvedDependencies,
+    ):
         self._search = search
         self._store = store
         self._resolved_deps = resolved_deps
@@ -831,7 +873,7 @@ class PackageBuilder:
         """
         Creates a new chroot and clones all PKGBUILDS.
         """
-        l.print_summary("Creating a build environment..")
+        l.print_info("Creating a build environment..")
 
         if os.path.exists(conf.build_dir):
             l.print_info("Removing previous build directory.")
@@ -847,7 +889,8 @@ class PackageBuilder:
             os.chdir(pkgbuild_dir)
 
             git_url_info = self._search.get_package_info(
-                self._resolved_deps.get_some_pkgname(pkgbase))
+                self._resolved_deps.get_some_pkgname(pkgbase)
+            )
 
             # Because all dependencies and packages should be resolved during the creation
             # of ResolvedDependencies. git_url should not be None.
@@ -858,7 +901,7 @@ class PackageBuilder:
             self._git_clone_and_review_pkgbuild(pkgbase, git_url)
             shutil.chown(pkgbuild_dir, user=conf.makepkg_user)
 
-        l.print_summary("Creating a new chroot.")
+        l.print_info("Creating a new chroot.")
         os.makedirs(self.chroot_wd_dir)
 
         # Remove GNUPGHOME from mkarchroot environment variables since it may interfere with
@@ -866,16 +909,16 @@ class PackageBuilder:
         mkarchroot_env_vars = os.environ.copy()
         try:
             del mkarchroot_env_vars["GNUPGHOME"]
-            l.print_debug(
+            l.print_debug("Removed GNUPGHOME variable from mkarchroot environment.")
-                "Removed GNUPGHOME variable from mkarchroot environment.")
         except KeyError:
             pass
 
-        subprocess.run(conf.commands.make_chroot(self.chroot_dir,
+        subprocess.run(
-                                                 list(self._pkgs_in_chroot)),
+            conf.commands.make_chroot(self.chroot_dir, list(self._pkgs_in_chroot)),
             env=mkarchroot_env_vars,
             check=True,
-                       capture_output=conf.suppress_command_output)
+            capture_output=conf.suppress_command_output,
+        )
 
     def remove_build_environment(self):
         """
@@ -883,8 +926,9 @@ class PackageBuilder:
         """
         shutil.rmtree(conf.build_dir)
 
-    def build_packages(self, package_base: str, packages: list[ForeignPackage],
+    def build_packages(
-                       force: bool):
+        self, package_base: str, packages: list[ForeignPackage], force: bool
+    ):
         """
         Builds package(s) with the same package base.
 
@@ -896,15 +940,14 @@ class PackageBuilder:
         # Rebuild is only needed if at least one package is not in the cache.
 
         if self._are_all_pkgs_cached(packages) and not force:
-            l.print_summary(
+            l.print_info(
                 f"Skipped building '{' '.join(package_names)}'. Already up to date."
             )
             return
 
-        l.print_summary(f"Building '{' '.join(package_names)}'.")
+        l.print_info(f"Building '{' '.join(package_names)}'.")
 
-        chroot_new_pacman_pkgs, chroot_pkg_files = self._get_chroot_packages(
+        chroot_new_pacman_pkgs, chroot_pkg_files = self._get_chroot_packages(packages)
-            packages)
 
         pkgbuild_dir = self.pkgbase_dir_map[package_base]
         os.chdir(pkgbuild_dir)
@@ -915,19 +958,24 @@ class PackageBuilder:
 
         l.print_info("Installing build dependencies to chroot.")
 
-        subprocess.run(conf.commands.install_chroot_packages(
+        subprocess.run(
+            conf.commands.install_chroot_packages(
                 self.chroot_dir,
-            chroot_new_pacman_pkgs + PackageBuilder.always_included_packages),
+                chroot_new_pacman_pkgs + PackageBuilder.always_included_packages,
+            ),
             check=True,
-                       capture_output=conf.suppress_command_output)
+            capture_output=conf.suppress_command_output,
+        )
 
         l.print_info("Making package.")
 
-        subprocess.run(conf.commands.make_chroot_pkg(self.chroot_wd_dir,
+        subprocess.run(
-                                                     conf.makepkg_user,
+            conf.commands.make_chroot_pkg(
-                                                     chroot_pkg_files),
+                self.chroot_wd_dir, conf.makepkg_user, chroot_pkg_files
+            ),
             check=True,
-                       capture_output=conf.quiet_output)
+            capture_output=conf.quiet_output,
+        )
 
         for pkgname in package_names:
             file = self._find_pkgfile(pkgname, pkgbuild_dir)
@@ -949,18 +997,27 @@ class PackageBuilder:
 
         l.print_info("Removing build dependencies from chroot.")
 
-        # FIX: If installed packages are virtual packages, removing them wont succeed.
         if len(chroot_new_pacman_pkgs) != 0:
             to_remove = []
             for p in chroot_new_pacman_pkgs:
                 if p not in self._pkgs_in_chroot:
-                    to_remove.append(strip_dependency(p))
+                    real_pkgname = (
-            subprocess.run(conf.commands.remove_chroot_packages(
+                        subprocess.run(
-                self.chroot_dir, to_remove),
+                            conf.commands.resolve_real_name(self.chroot_dir, p),
                             check=True,
-                           capture_output=conf.suppress_command_output)
+                            stdout=subprocess.PIPE,
+                        )
+                        .stdout.decode()
+                        .strip()
+                    )
+                    to_remove.append(real_pkgname)
+            subprocess.run(
+                conf.commands.remove_chroot_packages(self.chroot_dir, to_remove),
+                check=True,
+                capture_output=conf.suppress_command_output,
+            )
 
-        l.print_summary(f"Finished building: '{' '.join(package_names)}'.")
+        l.print_info(f"Finished building: '{' '.join(package_names)}'.")
 
     def _are_all_pkgs_cached(self, pkgs: list[ForeignPackage]) -> bool:
         for pkg in pkgs:
@@ -1027,8 +1084,10 @@ class PackageBuilder:
 
         for foreign_pkg in chroot_foreign_pkgs:
             entry = self._store.get_package(foreign_pkg)
-            assert entry is not None, "Build order determines that the dependencies are built \
+            assert entry is not None, (
+                "Build order determines that the dependencies are built \
 before and thus are found in the cache."
+            )
 
             _, file = entry
 
@@ -1067,32 +1126,55 @@ before and thus are found in the cache."
         The user is prompted to review the PKGBUILD and confirm if the package should be built.
         """
         try:
-            subprocess.run(conf.commands.git_clone(git_url, "."),
+            subprocess.run(
+                conf.commands.git_clone(git_url, "."),
                 check=True,
-                           capture_output=conf.suppress_command_output)
+                capture_output=conf.suppress_command_output,
+            )
 
-            if l.prompt_confirm(f"Review PKGBUILD for {pkgbase}?",
+            if l.prompt_confirm(
-                                default=True):
+                f"Review PKGBUILD or show diff for {pkgbase}?", default=True
-                latest_reviewed_commit = self._store.pkgbuild_latest_reviewed_commits.get(
+            ):
-                    pkgbase)
+                latest_reviewed_commit = (
-                if latest_reviewed_commit is None:
+                    self._store.pkgbuild_latest_reviewed_commits.get(pkgbase)
+                )
+
+                git_commit_ids = (
+                    subprocess.run(
+                        conf.commands.git_log_commit_ids(),
+                        check=True,
+                        stdout=subprocess.PIPE,
+                    )
+                    .stdout.decode()
+                    .strip()
+                    .split("\n")
+                )
+
+                if (
+                    latest_reviewed_commit is None
+                    or latest_reviewed_commit not in git_commit_ids
+                ):
                     for file in os.scandir("."):
                         if file.is_file() and not file.name.startswith("."):
-                            subprocess.run(conf.commands.review_file(
+                            subprocess.run(
-                                file.path),
+                                conf.commands.review_file(file.path), check=True
-                                           check=True)
+                            )
                 else:
                     subprocess.run(
-                        conf.commands.git_diff(latest_reviewed_commit),
+                        conf.commands.git_diff(latest_reviewed_commit), check=True
-                        check=True)
+                    )
 
             if l.prompt_confirm("Build this package?", default=True):
-                commit_id = subprocess.run(
+                commit_id = (
+                    subprocess.run(
                         conf.commands.git_get_commit_id(),
                         check=True,
-                    capture_output=True).stdout.decode().strip()
+                        capture_output=True,
-                self._store.pkgbuild_latest_reviewed_commits[
+                    )
-                    pkgbase] = commit_id
+                    .stdout.decode()
+                    .strip()
+                )
+                self._store.pkgbuild_latest_reviewed_commits[pkgbase] = commit_id
             else:
                 raise err.UserFacingError("Building aborted.")
 

tests/test_source_resolution.py

@@ -264,3 +264,49 @@ class TestSource(unittest.TestCase):
             self.source.packages_to_remove(self.currently_installed_packages),
             ["p4", "A4", "M_A1", "M_A2"],
         )
+
+class TestModuleUserServices(unittest.TestCase):
+
+    class ModuleWithUserServiceOne(Module):
+
+        def __init__(self):
+            super().__init__("one", True, "0")
+
+        def systemd_user_units(self) -> dict[str, list[str]]:
+            return {
+                    "user": ['foo.service']
+                    }
+
+    class ModuleWithUserServiceTwo(Module):
+
+        def __init__(self):
+            super().__init__("two", True, "0")
+
+        def systemd_user_units(self) -> dict[str, list[str]]:
+            return {
+                    "user": ['bar.service']
+                    }
+
+    def setUp(self) -> None:
+        self.source = Source(
+                pacman_packages=set(),
+                aur_packages=set(),
+                user_packages=set(),
+                ignored_packages=set(),
+                systemd_units=set(),
+                systemd_user_units={},
+                files={},
+                directories={},
+                modules={                
+                         self.ModuleWithUserServiceOne(),
+                         self.ModuleWithUserServiceTwo()
+                         },
+            )
+        self.store = Store()
+
+
+    def test_user_units_to_enable(self):
+        self.assertDictEqual(
+            self.source.user_units_to_enable(self.store),
+            {"user": ["foo.service", "bar.service"]},
+        )