117 Commits
Author SHA1 Message Date
Kivi Kaitaniemi 7bbd8a2cd8 Release v1.2.0 2026-02-16 21:53:00 +02:00
Kivi Kaitaniemi 55ad44b43c Merge pull request #57 from kiviktnm/feat/symlink-parent-folders
Add option to configure symlink initial parent folder permissions
2026-02-16 21:50:24 +02:00
Kivi Kaitaniemi a4ab83493f Add option to configure symlink initial parent folder permissions 2026-02-16 21:49:16 +02:00
Kivi Kaitaniemi 71a7e867ad Enhance README with user management and syntax update
Updated README to include user, group, and PGP key management information and modified package management syntax.
2026-02-08 12:06:58 +02:00
Kivi Kaitaniemi eafe7e7af9 Release v1.1.0 2026-02-07 20:58:05 +02:00
Kivi Kaitaniemi 5ac95facc3 Merge pull request #52 from kiviktnm/feat/run-multiple-methods-with-attr
Allow decorating multiple methods per module #47
2026-02-07 20:55:06 +02:00
Kivi Kaitaniemi 37745d1730 Allow decorating multiple methods per module #47 2026-02-07 20:52:32 +02:00
Kivi Kaitaniemi 7aa063f6b9 Merge pull request #51 from kiviktnm/fix/consider-provides
Fix/consider provides
2026-02-07 20:28:41 +02:00
Kivi Kaitaniemi 4203d2d439 Fix foreign upgrade after removal 2026-02-07 20:26:36 +02:00
Kivi Kaitaniemi c5898d82a3 Consider foreign provided packages when installing foreign packages #46 2026-02-07 20:24:29 +02:00
Kivi Kaitaniemi cdbf1e8348 Merge pull request #50 from kiviktnm/feat/print-failed-command-exit-code
Include exit code in CommandFailedError
2026-02-07 13:26:01 +02:00
Kivi Kaitaniemi e4eb81bd5b Include exit code in CommandFailedError 2026-02-07 13:25:33 +02:00
Kivi Kaitaniemi 97e94258a4 Merge pull request #49 from kiviktnm/fix/copy-local-pkgbuild-dir
Fix installing CustomPackages with 'install=' in PKGBUILD
2026-02-07 13:14:34 +02:00
Kivi Kaitaniemi 1214c69fba Fix installing CustomPackages with 'install=' in PKGBUILD #45 2026-02-07 13:13:14 +02:00
Kivi Kaitaniemi 1c0cbf3a53 Release v1.0.0 2026-01-27 23:47:36 +02:00
Kivi Kaitaniemi 928d7c4965 Don't try to remove empty build dependencies from chroot #44 2026-01-27 23:45:56 +02:00
Kivi Kaitaniemi 360e0fcf9e Add symlink support 2026-01-07 00:52:18 +02:00
Kivi Kaitaniemi 8ca2ca8a06 Don't fail on systemd disable failure 2026-01-07 00:03:57 +02:00
Kivi Kaitaniemi a5ec25693e Fix source and target ordering on file install errors 2026-01-06 20:08:54 +02:00
Kivi Kaitaniemi 04770f6a34 Update README.md 2025-12-29 18:13:20 +02:00
Kivi Kaitaniemi f8a6724816 Update README.md 2025-12-29 18:10:51 +02:00
Kivi Kaitaniemi b1dccf732b Improve docs 2025-12-28 09:27:49 +02:00
Kivi Kaitaniemi 4b5b703203 Add user and pgp key management modules (fixes #38) 2025-12-28 09:18:37 +02:00
Kivi Kaitaniemi 9b3454f434 decman.modules should be a list 2025-12-28 05:02:17 +02:00
Kivi Kaitaniemi 7e0b49be2f Improve orphan package discovery 2025-12-27 23:44:35 +02:00
Kivi Kaitaniemi 6078968ac1 Update license information 2025-12-27 04:18:08 +02:00
Kivi Kaitaniemi 041c976552 Rename packages to plugins 2025-12-27 04:04:04 +02:00
Kivi Kaitaniemi 9b2fc79829 Merge pull request #41 from kiviktnm/split-packages
Split plugins to seperate packages
2025-12-27 04:01:02 +02:00
Kivi Kaitaniemi f05f502fa7 Split plugins to seperate packages 2025-12-27 03:58:42 +02:00
Kivi Kaitaniemi 77bd72575e Use pyalpm instead of running pacman multiple times (fixes #39) 2025-12-27 02:43:02 +02:00
Kivi Kaitaniemi d54fd89852 Improve command output printing 2025-12-26 18:55:01 +02:00
Kivi Kaitaniemi f0a82dfa2d Don't print 'Pacman output highlights' unless there actually are highlights #40 2025-12-26 15:14:59 +02:00
Kivi Kaitaniemi d050372a33 When building custom packages, copy the whole PKGBUILD directory 2025-12-26 15:00:43 +02:00
Kivi Kaitaniemi 978333fcdb Handle KeyboardInterrupt 2025-12-21 04:16:46 +02:00
Kivi Kaitaniemi 5d4065623a Fix flatpak list_apps when no apps are installed 2025-12-19 23:05:42 +02:00
Kivi Kaitaniemi 3bbd5bd521 Fix issue with finding disabled modules 2025-12-18 23:06:22 +02:00
Kivi Kaitaniemi b2494c62d3 Merge pull request #37 from kiviktnm/new-structure
New architecture for decman
2025-12-18 22:46:11 +02:00
Kivi Kaitaniemi dd192d5871 Use print_command_output for CommandFailedError output 2025-12-18 22:44:27 +02:00
Kivi Kaitaniemi d084f826f1 Print method run as debug 2025-12-18 22:30:10 +02:00
Kivi Kaitaniemi bbbc6fdf7e Add missing example files 2025-12-18 06:51:35 +02:00
Kivi Kaitaniemi 3470efefb8 Merge branch 'main' into new-structure 2025-12-18 06:45:56 +02:00
Kivi Kaitaniemi 1f7982130f Set version to 1.0.0 2025-12-18 06:21:31 +02:00
Kivi Kaitaniemi 6b67fc0e53 Add migration steps 2025-12-18 06:20:58 +02:00
Kivi Kaitaniemi 36053c6d3b Fix ZSH completions 2025-12-18 06:13:01 +02:00
Kivi Kaitaniemi 45e8879962 Add shell completions for decman 2025-12-18 05:59:41 +02:00
Kivi Kaitaniemi 1e3b83b3d8 Add an example for using decman 2025-12-18 05:45:07 +02:00
Kivi Kaitaniemi 20171333ae Add missing flatpak plugin to entry points 2025-12-18 04:59:34 +02:00
Kivi Kaitaniemi 3baa5e032b Fix issues with tests and PKGBUILD reviewing 2025-12-18 04:00:36 +02:00
Kivi Kaitaniemi bed2b5eb3e Use nobody-user for parsing PKGBUILDs 2025-12-18 03:42:04 +02:00
Kivi Kaitaniemi 9ef1f23a64 Fix issues in the migration guide 2025-12-18 01:41:28 +02:00
Kivi Kaitaniemi d8ec223fc3 Add migration guide 2025-12-18 01:29:20 +02:00
Kivi Kaitaniemi 6a1f64c3dd Improve printed output 2025-12-17 21:54:53 +02:00
Kivi Kaitaniemi 757edfe7cc Add plugin documentation 2025-12-17 19:48:01 +02:00
Kivi Kaitaniemi c2bb67dc4e Start writing documentation for changes 2025-12-15 04:48:38 +02:00
Kivi Kaitaniemi c036ec03cc Add flatpak as a plugin 2025-12-14 23:43:09 +02:00
Kivi Kaitaniemi 6fe1f4cb6f Add AUR plugin 2025-12-14 22:28:08 +02:00
Kivi Kaitaniemi 5861ca219b Cache calls to PackageInfo 2025-12-14 19:22:06 +02:00
Kivi Kaitaniemi 5dd6d8b281 Implement PKGBUILD parsing for CustomPackages 2025-12-14 19:10:39 +02:00
Kivi Kaitaniemi 837d32ac43 Release v0.4.2 2025-12-14 17:14:35 +02:00
Kivi Kaitaniemi fb578c0511 Initial pacman code 2025-12-14 17:12:25 +02:00
Kivi Kaitaniemi 1372e17f4e Merge pull request #36 from goajos/main
flatpak bug v0.4.1 fix: https://github.com/kiviktnm/decman/issues/35
2025-12-14 17:11:26 +02:00
goajos 9b10819823 flatpak bug v0.4.1 fix: https://github.com/kiviktnm/decman/issues/35 2025-12-14 00:43:21 +01:00
Kivi Kaitaniemi ebf4b8e5ac Allow storing sets in Store 2025-12-13 20:58:24 +02:00
Kivi Kaitaniemi fd96cbec57 Implement systemd plugin 2025-12-13 20:51:21 +02:00
Kivi Kaitaniemi 9471bbf6ac Refactor main app 2025-12-13 17:35:21 +02:00
Kivi Kaitaniemi 364bf3d7b9 Add main app logic 2025-12-13 07:03:16 +02:00
Kivi Kaitaniemi 7c34b3c7d2 Change error parameter ordering 2025-12-13 04:42:08 +02:00
Kivi Kaitaniemi 786968feac Implement file installation 2025-12-13 04:38:58 +02:00
Kivi Kaitaniemi aaa4a0ad43 Add plugin architecture 2025-12-13 01:44:41 +02:00
Kivi Kaitaniemi 6047cad690 Unify docstring formats 2025-12-12 23:17:18 +02:00
Kivi Kaitaniemi 2b1bbdb884 Better methods for running commands 2025-12-12 23:08:30 +02:00
Kivi Kaitaniemi 9da024c8bd Migrate to uv 2025-12-12 17:03:15 +02:00
Kivi Kaitaniemi a566adf546 Release v0.4.1 2025-12-12 14:37:18 +02:00
Kivi Kaitaniemi 616efa9629 Remove accidentally commited wheel 2025-12-12 14:34:30 +02:00
Kivi Kaitaniemi 409146cf14 Release v0.4.0 2025-12-10 00:34:57 +02:00
Kivi Kaitaniemi 70306ce2ac Merge branch 'gro-david-main' 2025-12-10 00:30:09 +02:00
Kivi Kaitaniemi 2378f49676 Run flatpak commands on users with UID>=1000 2025-12-10 00:29:15 +02:00
Kivi Kaitaniemi a54fad6570 Merge branch 'main' of https://github.com/gro-david/decman into gro-david-main 2025-12-09 11:54:45 +02:00
Dávid Groniewsky 6ae8c78959 Switched the variable type of user defined flatpak packages to match the type of the user systemd services 2025-10-13 09:36:46 +02:00
David Groniewsky 69bcaadf90 Forgot to set the HOME variable so set that as suggested in the change request 2025-10-09 22:47:35 +02:00
David Groniewsky 7ea1ef2d81 Fixed the management of flatpak packages installed to the user installation as requested in the PR 2025-10-09 22:42:19 +02:00
David Groniewsky d765eea3f4 Simple fixes 2025-10-08 19:13:08 +02:00
Kivi Kaitaniemi 37e85a0814 Fix test that occasionally fails due to set ordering being indeterminate 2025-10-02 00:38:13 +03:00
Dávid Groniewsky 7320c88940 Updated the README and the examples to include user wide installation of flatpak packages. 2025-10-01 11:19:04 +02:00
Dávid Groniewsky 613beaa917 Added the option for installing flatpaks to the user installation. 2025-10-01 11:15:26 +02:00
Dávid Groniewsky 515a27cb7d Added the check for missing flatpak installation while the flatpaks are enabled. An error will be printed and decman will exit. 2025-09-30 22:45:54 +02:00
Dávid Groniewsky dbeed0b7d8 Updated the README with the announcement that flatpaks can now be managed. Added the required documentation to the example. 2025-09-30 22:40:19 +02:00
Dávid Groniewsky 9ce42ccf3e Added the config option to disable flatpaks (disabled by default), and the flag --no-flatpaks which will skip flatpak packages. 2025-09-30 22:30:24 +02:00
Dávid Groniewsky 0ecd835090 Extended the flatpak functionality to modules. All works as expected. 2025-09-29 15:19:42 +02:00
Dávid Groniewsky 27a8c279c4 Removed the build and dist directories 2025-09-29 14:58:21 +02:00
Dávid Groniewsky 401233a352 Added support for declaring installed and ignored flatpaks in the source file. For this the variables 'flatpak_packages' and 'ignored_flatpak_packages' are used. These work like the pacman equivalents. There were test cases appended for flatpak testing. I have installed the modified version on my system and tested it extensively. No errors were found. 2025-09-29 14:57:34 +02:00
Kivi Kaitaniemi f3910a6bc9 Release v0.3.4 2025-05-01 17:13:02 +03:00
Kivi Kaitaniemi bce9be5ebc Use pty for capturing and echoing cmd output 2025-05-01 17:10:16 +03:00
Kivi Kaitaniemi ec19b21244 Mention package groups in README.md 2025-05-01 16:49:06 +03:00
Kivi Kaitaniemi 47ea816d3f Update README.md & examples 2025-03-08 19:07:04 +02:00
Kivi Kaitaniemi 23620e86f5 Release v0.3.3 2025-03-08 17:30:45 +02:00
Kivi Kaitaniemi ef8423d128 Update README.md 2025-03-08 17:27:47 +02:00
Kivi Kaitaniemi db46ac339c Merge pull request #24 from kiviktnm/cache-aur-providers
Cache AUR package providers #23
2025-03-08 17:20:14 +02:00
Kivi Kaitaniemi 8e3646a5f0 Cache AUR package providers #23 2025-03-08 17:19:39 +02:00
Kivi Kaitaniemi cad1c2682c Merge pull request #22 from kiviktnm/fix-remove-chroot-virtual-pkgs
Fix removing virtual packages from chroots #7
2025-03-08 15:11:32 +02:00
Kivi Kaitaniemi 5ede2a355a Fix removing virtual packages from chroots #7 2025-03-08 15:09:39 +02:00
Kivi Kaitaniemi c6f180fb94 Merge pull request #15 from oatmealraisin/formatters
Add ruff and isort
2025-01-25 12:42:14 +02:00
Kivi Kaitaniemi 82a1fbdcb9 Merge pull request #16 from oatmealraisin/user_services
Fix bug in module user systemd unit handling
2025-01-25 12:21:03 +02:00
user e153b8c400 Add test for multiple module user services 2025-01-23 18:37:38 -05:00
user bde9d9c70d Fix bug in module user systemd unit handling 2025-01-23 18:10:36 -05:00
user f7567890da Add ruff and isort 2025-01-23 15:29:12 -05:00
Kivi Kaitaniemi 53b227859c Remind the user that systemd services won't be started/stopped 2025-01-19 11:23:06 +02:00
Kivi Kaitaniemi c091d18ee1 Prevent BlockingOIError from being raised when echonig large amounts of output 2024-07-28 16:01:46 +03:00
Kivi Kaitaniemi 7dacad6be2 Release v0.3.1 2024-07-28 11:19:28 +03:00
Kivi Kaitaniemi 5fe7df23db Show full PKGBUILD if a commit id is not found
If a git commit id is not found in a PKGBUILD repository, show the full
PKGBUILD instead of trying to show a git diff.

Fixes #5
2024-07-28 11:10:46 +03:00
Kivi Kaitaniemi 483bf0d676 Release v0.3.0 2024-07-06 18:52:14 +03:00
Kivi Kaitaniemi eb2ad7b1eb Merge pull request #4 from kiviktnm/pacman-color-always-as-default
Set pacman output use colors by default
2024-07-06 18:45:52 +03:00
Kivi Kaitaniemi cf1ae2ec50 Set pacman output use colors by default 2024-07-06 18:45:10 +03:00
Kivi Kaitaniemi 3b91577d61 Change pacman output highlight keyword defaults 2024-07-06 18:34:21 +03:00
Kivi Kaitaniemi 539bb3bfad Merge pull request #3 from kiviktnm/highlight-pacman-output
Add feature to print pacman output highlights
2024-07-06 18:23:13 +03:00
Kivi Kaitaniemi ab53a37f9f Add feature to print pacman output highlights 2024-07-06 18:20:10 +03:00
Kivi Kaitaniemi 1f8be0593f Update README.md 2024-05-30 15:06:43 +03:00
87 changed files with 13416 additions and 3898 deletions
+2
View File
@@ -4,3 +4,5 @@ build/
*.egg-info/
venv/
.venv/
dist/
+72
View File
@@ -0,0 +1,72 @@
# Commands used in development
Before committing ensure all tests pass and format files.
## Running
Run decman as root to test all changes:
```sh
sudo uv run --all-packages decman
```
## Python shell
Running a python shell with all the packages.
```sh
sudo uv run --all-packages python
sudo uv run --exact --package decman python
```
## Testing
Run all unit tests (`-s` disables output capturing, needed for PTY test):
```sh
uv run --package decman pytest -s tests/
uv run --package decman-pacman pytest plugins/decman-pacman/tests/
uv run --package decman-systemd pytest plugins/decman-systemd/tests/
uv run --package decman-flatpak pytest plugins/decman-flatpak/tests/
```
## Formatting
Format all files:
```sh
uv run ruff format
```
## Linting
Run lints:
```sh
uv run ruff check
```
Apply fixes:
```sh
uv run ruff check --fix
```
## Installing the example plugin
```sh
uv pip install -e example/plugin/
```
Uninstalling:
```sh
uv pip uninstall decman-plugin-example
```
Making the plugin available/unavailable:
```sh
touch /tmp/example_plugin_available
rm /tmp/example_plugin_available
```
+231 -182
View File
@@ -1,12 +1,15 @@
# Decman
Decman is a declarative package & configuration manager for Arch Linux. It allows you to manage installed packages, your dotfiles, enabled systemd units, and run commands automatically. Your system is configured using python so your configuration can be very adaptive.
> Decman has breaking changes!
> Decman has undergone an architecture rewrite. The new architecture makes decman more expandable and maintainable.
>
> Migration guide is [here](/docs/migrate-to-v1.md).
If you want, you can also use decman with other configuration languages. See the [example with TOML later in this README](#decman-with-other-configuration-languages).
Decman is a declarative package & configuration manager for Arch Linux. It allows you to manage installed packages, your dotfiles, enabled systemd units, and run commands automatically. Your system is configured using Python so your configuration can be very adaptive.
## Overview
A complete example is available in the `example`-directory of this repository. It also serves as documentation so reading it is recommended.
[See the example for a quick tutorial.](/example/README.md)
To use decman, you need a source file that declares your system installation. I recommend you put this file in source control, for example in a git repository.
@@ -14,26 +17,28 @@ To use decman, you need a source file that declares your system installation. I
```py
import decman
from decman import File, Directory
# Declare installed packages
decman.packages += ["python", "git", "networkmanager", "ufw", "neovim"]
# Declare installed pacman packages
decman.pacman.packages |= {"base", "linux", "linux-firmware", "networkmanager", "ufw", "neovim"}
# Declare installed aur packages
decman.aur_packages += ["protonvpn"]
decman.aur.packages |= {"decman"}
# Declare configuration files
# Inline
decman.files["/etc/vconsole.conf"] = File(content="KEYMAP=us")
# From files within your repository
# From files within your source repository
# (full path here would be /home/user/config/dotfiles/pacman.conf)
decman.files["/etc/pacman.conf"] = File(source_file="./dotfiles/pacman.conf")
# Declare a whole directory
decman.directories["/home/user/.config/nvim"] = Directory(source_directory="./dotfiles/nvim",
owner="user")
# Ensure that a systemd unit is enabled.
decman.enabled_systemd_units += ["NetworkManager.service"]
decman.systemd.enabled_units |= {"NetworkManager.service"}
```
To better organize your system configuration, you can create modules.
@@ -41,17 +46,20 @@ To better organize your system configuration, you can create modules.
`/home/user/config/syncthing.py`:
```py
from decman import Module, prg
from decman import Module, Store, prg
from decman.plugins import pacman, systemd
# Your custom modules are child classes of the module class.
# They can override methods of the Module-class.
class Syncthing(Module):
def __init__(self):
super().__init__(name="syncthing", enabled=True, version="1")
super().__init__(name="syncthing")
def on_enable(self):
# Run code when a module is first enabled
def on_enable(self, store: Store):
# Note: store is a key-value store that will persist between decman runs.
# You can use it to store your own data as well. Here it is not needed.
# Call a program
prg(["ufw", "allow", "syncthing"])
@@ -59,17 +67,26 @@ class Syncthing(Module):
# Run any python code
print("Remember to setup syncthing with the browser UI!")
def on_disable(self):
# On disable is a special method, it will get executed when this module no longer exists.
# Therefore it must be static, take no parameters, and inline all imports.
# Imported modules should be available everywhere.
@staticmethod
def on_disable():
# Run code when a module is disabled
prg(["ufw", "deny", "syncthing"])
import decman
decman.prg(["ufw", "deny", "syncthing"])
def pacman_packages(self) -> list[str]:
# Packages part of this module
return ["syncthing"]
# Decorate a function with @pacman.packages to indicate it returns a set of pacman packages
# to be installed
@pacman.packages
def pacman_packages(self) -> set[str]:
return {"syncthing"}
def systemd_user_units(self) -> dict[str, list[str]]:
# Systemd units are declared in a similiar fashion
@systemd.user_units
def systemd_user_units(self) -> dict[str, set[str]]:
# Systemd user units part of this module
return {"user": ["syncthing.service"]}
return {"user": {"syncthing.service"}}
```
Then import your module in your main source file.
@@ -83,7 +100,7 @@ from syncthing import Syncthing
decman.modules += [Syncthing()]
```
Then run decman. Note that terminal colors cannot be disabled for decman.
Then run decman.
> [!WARNING]
> Decman runs as root. This means that your `source.py` will be executed as root as well.
@@ -104,202 +121,223 @@ Decman has some CLI options, to see them all run:
decman --help
```
For troubleshooting and submitting issues, you should use the `--debug` option.
```sh
sudo decman --debug
```
[See the complete documentation for using decman.](/docs/README.md)
## Installation
Clone the decman PKGBUILD:
```sh
git clone https://github.com/kiviktnm/decman-pkgbuild.git
git clone https://aur.archlinux.org/decman.git
```
Review the PKGBUILD and install it.
```sh
cd decman-pkgbuild
cd decman
makepkg -si
```
So far I have not created an AUR package for decman, because I'm not sure if other people would find decman useful.
Remember to add decman to its own configuration.
```py
import decman
decman.aur.packages |= {"decman"}
```
## What decman manages?
### Packages
Decman has built-in functionality for managing files and directories. Additionally decman manages system state using plugins. By default decman ships with the following plugins:
Decman can be used to install pacman packages. Decman will install all packages defined in the source and **remove** all packages not defined in the source. You can set packages to be ignored by decman, so that it won't install them nor remove them.
- [pacman](/docs/pacman.md)
- [systemd](/docs/systemd.md)
- [aur](/docs/aur.md)
- [flatpak](/docs/flatpak.md)
Additionally management of [users, groups and PGP keys](/docs/extras.md) is provided by built-in modules.
Plugins can be disabled if desired and flatpaks are disabled by default.
Please read the documentation to understand the functionality of those plugins in detail. Here are quick examples to show what the default plugins are capable of.
### Pacman
Pacman plugins manages native packages. Native packages can be installed from the pacman repositories. This plugin will never touch AUR packages.
```py
# Include both foreign and pacman packages here.
decman.ignored_packages += ["yay", "opendoas"]
import decman
# Packages that decman ensures are installed to the system
decman.pacman.packages |= {"firefox", "reflector"}
# These packages will never get installed or removed by decman.
decman.pacman.ignored_packages |= {"opendoas"}
```
### Foreign packages
### AUR
> [!NOTE]
> Building of foreign packages is not the primary function of decman. There are some issues that I may or may not fix.
> If you can't build a package using decman, consider adding it to `ignored_packages` and building it yourself.
> Building of AUR or custom packages is not the primary function of decman. There are some issues that I may or may not fix.
> If you can't build a package using decman, consider adding it to `decman.aur.ignored_packages` and building it yourself.
Decman can install AUR packages as well as user defined packages. Foreign packages are AUR and user packages combined.
Here is an example of a user package. Managing user packages is somewhat cumbersome as you have to declare their versions, dependencies and make dependencies manually. However, you probably won't install many user packages anyway.
AUR plugins manages foreign packages. Foreign packages are installed from the AUR or other sources. This plugin will never touch native packages.
```py
decman.user_packages.append(
decman.UserPackage(
pkgname="decman-git",
# Note, this example may not be up to date
provides=["decman"],
version="0.2.1",
dependencies=["python", "python-requests", "devtools", "pacman", "systemd", "git"],
make_dependencies=[
"python-setuptools", "python-build", "python-installer", "python-wheel"
],
git_url="https://github.com/kiviktnm/decman-pkgbuild.git",
))
import decman
from decman.plugins.aur import CustomPackage
# AUR Packages that decman ensures are installed to the system
decman.aur.packages |= {"android-studio", "fnm-bin"}
# These foreign packages will never get installed or removed by decman.
decman.aur.ignored_packages |= {"yay"}
# You can add packages from custom sources.
# Just add a package name and repository / directory containing a PKGBUILD
decman.aur.custom_packages |= {
CustomPackage("decman", git_url="https://github.com/kiviktnm/decman-pkgbuild.git"),
CustomPackage("my-own-package", pkgbuild_directory="/path/to/directory/"),
}
```
Building of foreign packages happens in a chroot. This creates some overhead, but ensures clean builds. By default the chroot is created to `/tmp/decman/build`. I recommend to use a tempfs for the `/tmp/` directory to speed up builds. Also make sure that the tempfs-partition is large enough. I recommend at least 6 GB.
Build packages are stored in a cache `/var/cache/decman`. By default decman keeps 3 most recent versions of all packages.
### Systemd units
> [!NOTE]
> Decman will only enable and disable systemd services. It will not start or stop them.
Decman can enable systemd services, system wide or for a specific user. Decman will enable all units defined in the source, and disable them when they are removed from the source. If a unit is not defined in the source, decman will not touch it.
### Files
Decman functions as a dotfile manager. It will install the defined files and directories to their destinations. You can set file permissions, owners as well as define variables that will be substituted in the installed files. Decman keeps track of all files it creates and when a file is no longer present in your source, it will be also removed from its destination. This helps with keeping your system clean. However, decman won't ever remove directories as they might contain files that weren't created by decman.
### Commands
Modules have 4 methods: `on_enable`, `on_disable`, `after_update` and `after_version_change`. These will be executed if the module is enabled, the module is disabled, after every update and after the version of the module has changed. You can use the helper functions `prg` and `sh` to run programs. These programs could for example be used to update packages managed by another package manager.
## Order of operations
When decman runs, it does the following things in this order.
1. Disable systemd units that are no longer in the source.
1. Create and update files.
1. Remove files no longer in the source.
1. Remove packages not defined in the source.
1. Upgrade packages.
- To upgrade foreign devel packages (eg. `*-git`) use the `--upgrade-devel` CLI option.
1. Install new packages.
1. Enable new systemd units.
1. Run commands:
1. `on_enable`
1. `after_version_change`
1. `on_disable`
1. `after_update`
Operations may be skipped with command line options.
## Decman with other configuration languages
Since decman uses Python files to declare your system, you can easily parse another configuration language in your Python source instead of declaring things directly in Python.
<details>
<summary>Here is a basic example using TOML.</summary>
To use TOML with Python, you need the `toml`-package. To install it in Arch Linux, install the `python-toml`-package.
This example doesn't allow you to use decman's modules or set decman's settings using TOML. You'll have to set them using Python. With this example you can use both TOML and Python if you want.
It would be possible to add support for TOML decman modules, but I don't think it would be worth the effort.
Write this in your decman source.
```py
import toml
import decman
TOML_CONFIG_FILE="/your/file/here.toml"
# System-wide units
decman.systemd.enabled_units |= {"NetworkManager.service"}
# These functions convert TOML tables to decman Files/Directories/UserPackages.
def toml_to_decman_file(toml_dict) -> decman.File:
return decman.File(content=toml_dict.get("content"),
source_file=toml_dict.get("source_file"),
bin_file=toml_dict.get("bin_file", False),
encoding=toml_dict.get("encoding", "utf-8"),
owner=toml_dict.get("owner"),
group=toml_dict.get("group"),
permissions=toml_dict.get("permissions", 0o644))
def toml_to_decman_directory(toml_dict) -> decman.Directory:
return decman.Directory(source_directory=toml_dict["source_directory"],
bin_files=toml_dict.get("bin_files", False),
encoding=toml_dict.get("encoding", "utf-8"),
owner=toml_dict.get("owner"),
group=toml_dict.get("group"),
permissions=toml_dict.get("permissions", 0o644))
def toml_to_decman_user_package(toml_dict) -> decman.UserPackage:
return decman.UserPackage(pkgname=toml_dict["pkgname"],
version=toml_dict["version"],
dependencies=toml_dict["dependencies"],
git_url=toml_dict["git_url"],
pkgbase=toml_dict.get("pkgbase"),
provides=toml_dict.get("provides"),
make_dependencies=toml_dict.get("make_dependencies"),
check_dependencies=toml_dict.get("check_dependencies"))
# Parse TOML into a Python dictionary
toml_source = toml.load(TOML_CONFIG_FILE)
# Set decman variables using the parsed dictionary.
decman.packages = toml_source.get("packages", [])
decman.aur_packages = toml_source.get("aur_packages", [])
decman.ignored_packages = toml_source.get("ignored_packages", [])
decman.enabled_systemd_units = toml_source.get("enabled_systemd_units", [])
decman.enabled_systemd_user_units = toml_source.get("enabled_systemd_user_units", {})
for filename, toml_file_dec in toml_source.get("files", {}).items():
decman.files[filename] = toml_to_decman_file(toml_file_dec)
for dirname, toml_dir_dec in toml_source.get("directories", {}).items():
decman.directories[dirname] = toml_to_decman_directory(toml_dir_dec)
for toml_user_package_dec in toml_source.get("user_packages", []):
decman.user_packages.append(toml_to_decman_user_package(toml_user_package_dec))
# User specific units
decman.systemd.enabled_user_units.setdefault("user", set()).update({"syncthing.service"})
```
Then you can use TOML configuration like this:
### Flatpak
```toml
packages = ["python", "git", "networkmanager", "ufw", "neovim", "python-toml"]
aur_packages = ["protonvpn"]
enabled_systemd_units = ["NetworkManager.service"]
ignored_packages = ["yay"]
user_packages = [{
pkgname="decman-git",
# Note, this example may not be up to date
provides=["decman"],
version="0.2.1",
dependencies=["python", "python-requests", "devtools", "pacman", "systemd", "git"],
make_dependencies=[
"python-setuptools", "python-build", "python-installer", "python-wheel"
],
git_url="https://github.com/kiviktnm/decman-pkgbuild.git"
}]
```py
import decman
[files]
'/etc/vconsole.conf' = { content="KEYMAP=us" }
'/etc/pacman.conf' = { source_file="./dotfiles/pacman.conf" }
# Flatpaks that decman ensures are installed to the system
decman.flatpak.packages |= {"org.mozilla.firefox", "org.signal.Signal"}
[directories]
'/home/user/.config/nvim' = { source_directory="./dotfiles/nvim", owner="user" }
# Flatpaks can be installed to specific users only
decman.flatpak.user_packages.setdefault("user", {}).update({"com.valvesoftware.Steam"})
# To set other file/directory attributes, just add them like this. Note that here I am not using octal notation for permissions.
'/home/user/.config/example' = { source_directory="./dotfiles/example", owner="user", group="user", bin_files=true, encoding="utf-8", permissions=448 }
[enabled_systemd_user_units]
user = ["syncthing.service"]
user2 = ["example.service", "another.service"]
# These flatpaks will never get installed or removed by decman.
decman.flatpak.ignored_packages |= {"dev.zed.Zed"}
```
</details>
### Users and PGP keys
Decman ships with built-in modules for managing users, groups and PGP keys. The modules don't support all features. In particular the PGP module is inteded only for AUR packages. However, they still allow managing users declaratively. Read more about them [here](/docs/extras.md).
Here these modules are used to create a `builduser` for AUR packages.
```python
import decman
import os
from decman.extras.gpg import GPGReceiver
from decman.extras.users import User, UserManager
um = UserManager()
gpg = GPGReceiver()
# Add a normal user
um.add_user(User(
username="alice",
groups=("libvirt"),
shell="/usr/bin/fish",
))
# Create builduser
um.add_user(User(
username="builduser",
home="/var/lib/builduser",
system=True,
))
# Receive desired PGP keys to that account (Spotify as an example)
gpg.fetch_key(
user="builduser",
gpg_home="/var/lib/builduser/gnupg",
fingerprint="E1096BCBFF6D418796DE78515384CE82BA52C83A",
uri="https://download.spotify.com/debian/pubkey_5384CE82BA52C83A.gpg",
)
# Configure aur to use builduser and the GNUPGHOME.
os.environ["GNUPGHOME"] = "/var/lib/builduser/gnupg"
decman.aur.makepkg_user = "builduser"
# Add version control systems required by the packages
decman.pacman.packages |= {"fossil"}
# Add AUR packages that require PGP keys or builduser setup
decman.aur.packages |= {"spotify", "pikchr-fossil"}
# Order matters here, users should be added before gpg keys
decman.modules += [um, gpg]
```
## Managing plugins and the order of operations
The order of operations is managed by setting `decman.execution_order`. This is also the default.
```py
import decman
decman.execution_order = [
"files",
"pacman",
"aur",
"systemd",
]
```
This variable also manages which plugins are enabled. To enable flatpaks, simply add the plugin to the execution order.
```py
import decman
decman.execution_order = [
"files",
"pacman",
"aur",
"flatpak",
"systemd",
]
```
Note that `files` is not a plugin, but is defined here anyways.
Before the core execution order, decman will run hook methods from `Module`s.
1. `before_update`
2. `on_disable`
After the plugin execution, decman will run the following hook methods.
1. `on_enable`
2. `on_change`
3. `atfer_update`
Operations and hooks may be skipped with command line options.
```sh
# Skip the aur plugin
sudo decman --skip aur
# Only apply file operations
sudo decman --no-hooks --only files
```
## Why use decman?
@@ -314,14 +352,16 @@ You can consult your config to see what packages are installed and what config f
In a modular config, you can also change parts of your system eg. switch shells without it affecting your other setups at all. If you create a module called `Shell` that exposes a function `add_alias`, you can call that function from other modules. Then later if you decide to switch from bash to fish, you can change the internals of your `Shell`-module without modifying your other modules at all.
```py
from decman import Module
# Look below for an example of a theme module
import theme
class Shell(Module):
def __init__(self):
super().__init__("shell", enabled=True, version="1")
super().__init__("shell")
self._aliases_text = ""
# --
def add_alias(self, alias: str, cmd: str):
self._aliases_text += f"alias {alias}='{cmd}'\n"
@@ -331,7 +371,6 @@ class Shell(Module):
File(source_file="./files/shell/config.fish", owner="user")
}
def file_variables(self) -> dict[str, str]:
fvars = {
"%aliases%": self._aliases_text,
@@ -367,24 +406,34 @@ Using python you can use the same config for different computers and only change
```py
import socket
import decman
if socket.gethostname() == "laptop":
# add brightness controls to your laptop
decman.packages += ["brightnessctl"]
decman.pacman.packages |= {"brightnessctl"}
```
## Alternatives
There are some alternatives you may want to consider instead of using decman.
- [Ansible](https://docs.ansible.com/)
- [aconfmgr](https://github.com/CyberShadow/aconfmgr)
- [NixOS](https://nixos.org/)
### Why not use NixOS?
NixOS is a Linux disto built around the idea of declarative system management, so why create a more limited alternative?
I tried NixOS in the past, but it had some issues that caused me to create decman for Arch Linux instead. In my personal opinion:
I tried NixOS in the past, but it had some issues that caused me to create decman for Arch Linux instead. In my opinion:
- NixOS forces you to do everything the Nix way. Sometimes I just want to develop software without having to use nix tools.
- NixOS is hard, and the documentation (when I last tried it) wasn't that good. Doing more complex stuff was sometimes just very annoying.
- NixOS has unnecessary abstraction with NixOS options. They are great until you have to configure something specific and there is not an option for it. Then you'll have to inline other configuration language within your Nix config. And if some software doesn't have any premade options you'll have to do write the config manually. Then you'll have some software managed with just options and others with normal config files. I prefer to keep everything consistent.
- NixOS forces you to do everything the Nix way.
- NixOS requires learning a new domain specific language.
- NixOS is extreme when it comes to declaration. Sometimes you don't want _everything_ to be managed declaratively.
## License
Copyright (C) 2024 Kivi Kaitaniemi
Copyright (C) 2024-2025 Kivi Kaitaniemi
Decman is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as
published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+14
View File
@@ -0,0 +1,14 @@
#compdef decman
# zsh completion for decman
_arguments -s \
'--source=[python file containing configuration]:config file:_files' \
'--dry-run[print what would happen as a result of running decman]' \
'--print[print what would happen as a result of running decman]' \
'--debug[show debug output]' \
'--skip[skip the following execution steps]:step(s):_message -r "step"' \
'--only[run only the following execution steps]:step(s):_message -r "step"' \
'--no-hooks[don'\''t run hook methods for modules]' \
'--no-color[don'\''t print messages with color]' \
'--params[additional parameters passed to plugins]:param(s):_message -r "param"' \
'--help[show help]'
+30
View File
@@ -0,0 +1,30 @@
# bash completion for decman
_decman() {
local cur prev opts
COMPREPLY=()
cur="${COMP_WORDS[COMP_CWORD]}"
prev="${COMP_WORDS[COMP_CWORD-1]}"
opts="--source --dry-run --print --debug --skip --only --no-hooks --no-color --params --help"
case "$prev" in
--source)
# file completion
COMPREPLY=( $(compgen -f -- "$cur") )
return 0
;;
--skip|--only|--params)
# free-form list
return 0
;;
esac
if [[ "$cur" == --* ]]; then
COMPREPLY=( $(compgen -W "$opts" -- "$cur") )
return 0
fi
return 0
}
complete -F _decman decman
+11
View File
@@ -0,0 +1,11 @@
# fish completion for decman
complete -c decman -l source -r -d "python file containing configuration" -a "(__fish_complete_path)"
complete -c decman -l dry-run -d "print what would happen as a result of running decman"
complete -c decman -l print -d "print what would happen as a result of running decman"
complete -c decman -l debug -d "show debug output"
complete -c decman -l skip -r -d "skip the following execution steps"
complete -c decman -l only -r -d "run only the following execution steps"
complete -c decman -l no-hooks -d "don't run hook methods for modules"
complete -c decman -l no-color -d "don't print messages with color"
complete -c decman -l params -r -d "additional parameters passed to plugins"
+567
View File
@@ -0,0 +1,567 @@
# Decman documentation
This contains the documentation for decman. Each plugin has its own documentation. For a quick overview of decman, see the [README](/README.md). For a tutorial, see the [example](/example/README.md).
- [pacman](/docs/pacman.md)
- [systemd](/docs/systemd.md)
- [aur](/docs/aur.md)
- [flatpak](/docs/flatpak.md)
Check out [extras](/docs/extras.md) for documentation for built-in modules.
## Quick notes
"Decman source" or "source" refers to your system configuration. It is set using the `--source` command line argument with decman.
```sh
sudo decman --source /this/is/your/source.py
```
Decman and decman plugins use sets for most collections to avoid duplicates. Remember to add values to sets instead of reassigning it.
```py
import decman
# GOOD
decman.pacman.packages |= {"vim"}
# BAD, now there is only "vim" in the packages, all previous operations were overridden.
decman.pacman.packages = {"vim"}
```
In Python you should not use from imports with global variables. It can lead to issues.
```py
# DO THIS:
import decman
decman.pacman.packages |= {"vim"}
# THIS MAY NOT WORK
from decman import pacman
pacman.packages |= {"vim"}
```
You can still import classes and functions with from imports safely.
```py
from decman import File
# pacman here refers to the pacman module containing the plugin
# not the plugin instance
from decman.plugins import pacman
```
## Decman Store
Decman stores data in the file `/var/lib/decman/store.json`. This file should not be modified manually. However, if encountering bugs with decman, manual modification may be desirable. The file is JSON so editing it should be easy enough.
Using the store:
```py
# The store is always given as a parameter to a method call.
# You don't need to create new instances.
store["key"] = value
# To ensure that a key exists (with a default value if it doesn't)
store.ensure("my_dict", {})
store["my_dict"]["dict_key"] = 3
```
This store is also available to plugins and modules. The following keys are used by decman:
- `allow_running_source_without_prompt`
- `source_file`
- `enabled_modules`
- `module_on_disable_scripts`
- `all_files`
Details about the keys used by each plugin are provided in the plugins documentation.
## Configuring decman
Decman has a small number of configuration options. They are set in your source file with python. These values are prioritized over command line options.
Import the config to modify it.
```py
import decman.config
```
Enable debug messages
```py
decman.config.debug_output = False
```
Disable info messages
```py
decman.config.quiet_output = False
```
Set colored output. This setting should not be used. It should be passed as a command line argument or an environment variable instead.
- Command line argument: `--no-color`
- Environment variables:
- `NO_COLOR`: disables color
- `FORCE_COLOR`: enables color
```py
decman.config.color_output = True
```
Directory for scripts containing Modules' on_disable code
```py
decman.config.module_on_disable_scripts_dir = "/var/lib/decman/scripts/"
```
Cache directory. Plugins like the AUR plugin use this directory as their own cache.
```py
decman.config.cache_dir = "/var/cache/decman"
```
The architecture of the computer's CPU. Currently, this is only used by the AUR plugin, but it may be useful for some other plugins.
```py
decman.config.arch = "x86_64"
```
## Files, directories and symlink
Decman functions as a dotfile manager. It will install the defined files, directories and symlinks to their destinations. You can set file permissions, owners as well as define variables that will be substituted in the installed files. Decman keeps track of all files it creates and when a file is no longer present in your source, it will be also removed from its destination. This helps with keeping your system clean. However, decman won't remove directories as they might contain files that weren't created by decman.
Symlinks management is simpler and more limited than for files since symlinks cannot have file permissions or ownership.
Variables can only be defined for files within modules. See the module example for using file variables.
Files and directories are updated during the `files` execution order step.
### File
Declarative file specification describing how a file should be materialized at a target path.
```py
from decman import File
import decman
# To declare a file, add it's target path and create a File object
decman.files["/home/me/.config/nvim/init.lua"] = File(
source_file="./dotfiles/nvim/init.lua",
bin_file=False,
encoding="utf-8",
owner="me",
group="users",
permissions=0o700,
)
```
Exactly one of `source_file` or `content` must be provided.
The file can be created by copying an existing source file or by writing provided content. For text files, optional variable substitution is applied at copy time. Binary files are copied or written verbatim and never undergo substitution.
Ownership, permissions, and parent directories are enforced on creation. Missing parent directories are created recursively and assigned the same ownership as the file when specified.
#### Parameters:
- `source_file: str`: Path to an existing file to copy from. Mutually exclusive with `content`.
- `content: str`: In-memory file contents to write. Mutually exclusive with `source_file`.
- `bin_file: bool`: If `True`, treat the file as binary. Disables variable substitution and writes bytes verbatim.
- `encoding: str`: Text encoding used when reading or writing non-binary files.
- `owner: str`: System user name to own the file and created parent directories.
- `group: str`: System group name to own the file and created parent directories. By default the `owner`'s group is used.
- `permissions: int`: File mode applied to the target file (e.g. `0o644`).
Note: Variable substitution is a simple string replacement where each key in variables is replaced by its corresponding value. No escaping or templating semantics are applied.
### Directory
Declarative specification for copying the contents of a source directory into a target directory.
```py
from decman import Directory
import decman
# To declare a directory, add it's target path and create a Directory object
decman.directories["/home/me/.config/nvim"] = File(
source_directory="./dotfiles/nvim",
bin_files=False,
encoding="utf-8",
owner="me",
group="users",
permissions=0o600,
)
```
For text files in the directory, optional variable substitution is applied at copy time. Binary files are copied or written verbatim and never undergo substitution.
Ownership, permissions, and parent directories are enforced on creation. Missing parent directories are created recursively and assigned the same ownership as the target directory when specified.
#### Parameters:
- `source_directory: str`: Path to the directory whose contents will be mirrored into the target.
- `bin_files: bool`: If `True`, treat all files as binary. Disables variable substitution and copies bytes verbatim.
- `encoding: str`: Text encoding used when reading or writing non-binary files.
- `owner: str`: System user name to own the files and directories.
- `group: str`: System group name to own the files and directories. By default the `owner`'s group is used.
- `permissions: int`: File mode applied to the created or updated files (e.g. `0o644`).
### Symlink
Declare a link to a target. Missing directories are created. If you need to configure parent folder permissions, use `Symlink` objects.
```py
import decman
# Replaces sudo with doas
# /usr/bin/sudo -> /usr/bin/doas
decman.symlinks["/usr/bin/sudo"] = "/usr/bin/doas"
# I don't know why would you ever do this but as an example
decman.symlinks["/home/me/.bin/mydoas"] = decman.Symlink("/usr/bin/doas", owner="me", group="users")
```
## Modules
Modules allow grouping related functionality together.
A **Module** is the primary unit for grouping related files, directories, packages, and executable logic in decman. Create your own modules by subclassing `Module`. Then override the methods documented below.
Each module is uniquely identified by its `name`.
Remember to add modules to decman. Modules are added to a list to preserve deterministic execution order for hooks. Modules added first will be executed first.
```py
import decman
decman.modules += [MyModule()]
```
### Basic Structure
```python
from decman import Module
class MyModule(Module):
def __init__(self) -> None:
super().__init__("my-module")
```
### Lifecycle Hooks
Modules can hook into specific phases of a decman run by overriding methods.
#### Before update
Executed **before** any updates are applied.
```python
def before_update(self, store):
...
```
#### After update
Executed **after** all updates are applied.
```python
def after_update(self, store):
...
```
#### On enable
Executed **once**, when the module transitions from disabled to enabled.
```python
def on_enable(self, store):
...
```
#### On change
Executed when the modules **content changes** between runs. Module's content is deemed changed if:
- If files or directories defined within the module have their content updated
- A plugin marks the module as changed
- For example, the pacman plugin marks a module as changed if the packages defined within that module change
```python
def on_change(self, store):
...
```
#### On disable
Executed when the module is disabled. A module is disabled when it's removed from the modules set.
**Must be declared as `@staticmethod`.**
Validated at class creation time.
```python
@staticmethod
def on_disable():
import os
os.remove("/some/file")
```
**Important constraints:**
- Code is copied verbatim into a temporary file
- No external variables
- Imports must be inside the function
- Signature must be exactly `on_disable()`
### Filesystem Declarations
Modules can declaratively define files and directories to be installed.
#### Files
Returns a mapping of target paths to `File` objects.
```python
def files(self) -> dict[str, File]:
return {
"/etc/myapp/config.conf": File(source_file="./dotfiles/config.conf"),
}
```
#### Directories
Returns a mapping of target paths to `Directory` objects.
```python
def directories(self) -> dict[str, Directory]:
return {
"/var/lib/myapp": Directory(source_directory="./dotfiles/myapp"),
}
```
#### File Variable Substitution
Defines variables that are substituted inside **text files** belonging to the module.
```python
def file_variables(self) -> dict[str, str]:
return {
"HOSTNAME": "example.com",
"PORT": "8080",
}
```
#### Symlinks
Defines symlinks fro the module.
```py
def symlinks(self) -> dict[str, str | Symlink]:
return {
"/etc/resolv.conf": "/run/systemd/resolve/resolv.conf",
"/home/me/.config/app/file.conf": Symlink("/home/me/.file.conf", owner="me"),
}
```
### Extending with plugins
To include plugin functionality inside a module, create a new method and mark it with the plugin's decorator. During the execution of decman, the plugin will call the marked method and use its result. Here is an example with the pacman plugin.
```py
from decman.plugins import pacman
@pacman.packages
def pacman_packages(self) -> set[str]:
return {"wget", "zip"}
```
## Plugins
Plugins are used to manage a single aspect of a system declaratively. Decman ships with some default plugins useful with Arch Linux but it is possible to add custom plugins.
To manage the execution order of plugins set `decman.execution_order`.
```py
import decman
decman.execution_order = [
"files", # not a plugin but included here
"pacman",
"aur",
"flatpak",
"systemd",
]
```
Available plugins are found in `decman.plugins`. You can add your own plugins to that dictionary.
```py
import decman
my_plugin = MyPlugin()
decman.plugins["my-plugin"] = my_plugin
# Remember to include your plugin in the execution order
decman.execution_order += ["my-plugin"]
```
For conveniance, decman provides some plugins with quick access.
```py
import decman
assert decman.pacman == decman.plugins.get("pacman")
assert decman.aur == decman.plugins.get("aur")
assert decman.systemd == decman.plugins.get("systemd")
assert decman.flatpak == decman.plugins.get("flatpak")
```
### Creating custom plugins
Create your own modules by subclassing `Plugin`. Then override the methods documented below.
#### Basic Structure
```python
from decman.plugins import Plugin
class MyPlugin(Plugin):
# Plugins should be singletons. (Only one instance exists ever.)
# This name should be the same as the key used in decman.plugins dict.
NAME = "my-plugin"
```
#### Availability check
Checks if this plugin can be enabled. For example, this could check if a required command is available. Returns `True` if this plugin can be enabled.
This is not useful if the plugin is directly added to `decman.plugins`. However, if using the Python package method for installing plugins, this check is used before adding the plugin automatically to `decman.plugins`.
Please note that this availibility check is executed before **any** decman steps. If a plugin depends on a pacman package, and that package is defined in the source but not yet installed, the plugin will not be available during the first run of decman.
```py
def available(self) -> bool:
return True
```
#### Process modules
This method gathers state information from modules. If the module's state has changed since the last time running this plugin, set the module to changed. For example, the pacman plugin uses this method to find which modules have methods marked with `@pacman.packages` and calls them.
This method only gathers information. It doesn't apply it.
```py
from decman import Store, Module
def process_modules(self, store: Store, modules: list[Module]):
...
# Toy example for setting modules as changed
for module in modules:
module._changed = True
```
#### Apply
Ensures that the state managed by this plugin is present on the system.
`dry_run` indicates that changes should only be printed, not yet applied.
`params` is a list of strings passed as command line arguments. For example running `decman --params abc def` would cause `params = ["abc", "def"]`.
This method must not raise exceptions. Instead it should return `False` to indicate a
failure. The method should handle it's exceptions and print them to the user.
```py
from decman import Store
def apply(
self, store: Store, dry_run: bool = False, params: list[str] | None = None
) -> bool:
return True
```
### Installing plugins as Python packages
You can have decman automatically detect plugins by creating a Python package with entry points in `decman.plugins`. Decman also does this with its own plugins.
In `pyproject.toml` set:
```toml
[project.entry-points."decman.plugins"]
pacman = "decman.plugins.pacman:Pacman"
aur = "decman.plugins.aur:AUR"
```
## Useful utilities
Decman ships with some useful utilites that can help with modules and plugins.
### Run commands
Runs a command and returns its output.
```py
import decman
decman.prg(
["nvim", "--headless", "+Lazy! sync", "+qa"],
user = "user",
env_overrides = {"EXAMPLE": "value"},
pass_environment = True,
mimic_login = True,
pty = True,
check = True,
)
```
#### Parameters
- `cmd: list[str]`: Command to execute.
- `user: str`: User name to run the command as. If set, the command is executed after dropping privileges to this user.
- `pass_environment: bool`: Copy decman's execution environment variables and pass them to the subprocess.
- `env_overrides: dict[str, str]`: Environment variables to override or add for the command execution. These values are merged on top of the current process environment.
- `mimic_login: bool`: If mimic_login is True, will set the following environment variables according to the given user's passwd file details. This only happens when user is set.
- `HOME`
- `USER`
- `LOGNAME`
- `SHELL`
- `pty: bool`: If `True`, run the command inside a pseudo-terminal (PTY). This enables interactive behavior and terminal-dependent programs. If `False`, run the command without a PTY using standard subprocess execution.
- `check`: If `True`, raise `decman.core.error.CommandFailedError` when the command exits with a non-zero status. If `False`, print a warning when encountering a non-zero exit code.
### Run a command in a shell
Runs a command in a shell and returns its output. Almost same as `decman.prg` but takes a string argument instead of a list and for example shell redirects are allowed.
```py
import decman
decman.sh(
"echo $EXAMPLE | less",
user = "user",
env_overrides = {"EXAMPLE": "value"},
mimic_login = True,
pty = True,
check = True,
)
```
#### Parameters
- `sh_cmd: str`: Shell command to execute.
- `user: str`: User name to run the command as. If set, the command is executed after dropping privileges to this user.
- `env_overrides dict[str, str]`: Environment variables to override or add for the command execution. These values are merged on top of the current process environment.
- `mimic_login: bool`: If mimic_login is True, will set the following environment variables according to the given user's passwd file details. This only happens when user is set.
- `HOME`
- `USER`
- `LOGNAME`
- `SHELL`
- `pty: bool`: If `True`, run the command inside a pseudo-terminal (PTY). This enables interactive behavior and terminal-dependent programs. If `False`, run the command without a PTY using standard subprocess execution.
- `check`: If `True`, raise `decman.core.error.CommandFailedError` when the command exits with a non-zero status. If `False`, print a warning when encountering a non-zero exit code.
### Errors
When your source needs to raise an error, decman provides `SourceError`s. Running commands with `prg` and `sh` may raise `decman.core.error.CommandFailedError`s if `check` is set to `True`. These are the errors that should be raised when decman runs your `source.py` file.
```py
import decman
raise decman.SourceError("boom")
```
### Decman Core
Additionally, you can import the modules used by decman. They should be relatively stable and not change too much between decman versions. The module `decman.core.output` is probably the most relevant one, as it provides methods for printing output.
+307
View File
@@ -0,0 +1,307 @@
# AUR
> [!NOTE]
> While this plugin exists with the sole purpose of installing foreing packages, this functionality is not the primary purpose of decman. Issues regarding this plugin are not a priority.
> If you can't build a package with this plugin, consider adding it to `ignored_packages` and building it yourself.
AUR plugin can be used to manage AUR and custom packages. The pacman plugin manages only foreing packages installed from the AUR or elsewhere. All native (pacman repositories) packages are ignored by this plugin with the exception that this plugin will install native dependencies of foreign packages.
It manages packages exactly the same way as the pacman plugin.
> This plugin will ensure that explicitly installed packages match those defined in the decman source. If your system has explicitly installed package A, but it is not included in the source, it will be uninstalled. You don't need to list dependencies in your source as those will be handeled by pacman automatically. However, if you have inluded package B in your source and that package depends on A, this plugin will not remove A. Instead it will demote A to a dependency. This plugin will also remove all orphaned packages automatically. **Packages that are only optionally required by other packages are considered orphans.** This way this plugin can ensure that your system truly matches your source. You cannot install an optional dependency, and forget about it later.
Building of foreign packages happens in a chroot. This creates some overhead, but ensures clean builds. By default the chroot is created to `/tmp/decman/build`. If `/tmp` is a in-memory filesystem like tmpfs, make sure that the tmpfs-partition is large enough. I recommend at least 6 GB. You can also change the build directory if memory is an issue.
Build packages are by default stored in a cache `/var/cache/decman/aur`. This plugin keeps 3 most recent versions of all packages.
When installing packages from other version control systems than git, you'll need to install the package for that VCS.
## Usage
Define AUR packages. These will be installed from the AUR.
```py
import decman
decman.aur.packages |= {"android-studio", "fnm-bin"}
```
Define ignored foreing packages. These can be AUR packages or other foreign packages. These packages will never get installed or removed by the plugin.
```py
decman.aur.ignored_packages |= {"yay"}
```
Define packages from custom sources. Add a package name and repository / directory containing a PKGBUILD. This plugin will fetch the PKGBUILD, generate .SRCINFO and parse that to find the package details.
```py
from decman.plugins.aur import CustomPackage
decman.aur.custom_packages |= {
CustomPackage("decman", git_url="https://github.com/kiviktnm/decman-pkgbuild.git"),
CustomPackage("my-own-package", pkgbuild_directory="/path/to/directory/"),
}
```
This plugin's execution order step name is `aur`.
### Command line
This plugin accepts params via the command line.
```sh
sudo decman --params aur-upgrade-devel aur-force
```
`aur-upgrade-devel` causes devel packages (packages from version control, such as `*-git` packages) to be upgraded.
`aur-force` causes decman to rebuild packages that were already cached.
### Within modules
Modules can also define AUR packages and custom packages. Decorate a module's method with `@decman.plugins.aur.packages` or `@decman.plugins.aur.custom_packages`. For AUR packages return a `set[str]` of package names from that module. Custom packages should return a `set[CustomPackage]`.
```py
import decman
from decman.plugins import aur
class MyModule(decman.Module):
...
@aur.packages
def aur_packages_defined_in_this_module(self) -> set[str]:
return {"android-studio", "fnm-bin"}
@aur.custom_packages
def custom_packages_defined_in_this_module(self) -> set[aur.CustomPackage]:
return {
CustomPackage("decman", git_url="https://github.com/kiviktnm/decman-pkgbuild.git"),
}
```
If these sets change, this plugin will flag the module as changed. The module's `on_change` method will be executed.
## Recommended setup
I recommend setting up a build user for AUR packages. Then you can import PGP keys to that user's keyring that will be used for verifying AUR packages. The build user setup might help with some version control systems such as fossil packages.
```python
import decman
import os
from decman.extras.gpg import GPGReceiver
from decman.extras.users import User, UserManager
um = UserManager()
gpg = GPGReceiver()
# Create builduser
um.add_user(User(
username="builduser",
home="/var/lib/builduser",
system=True,
))
# Receive desired PGP keys to that account (Spotify as an example)
gpg.fetch_key(
user="builduser",
gpg_home="/var/lib/builduser/gnupg",
fingerprint="E1096BCBFF6D418796DE78515384CE82BA52C83A",
uri="https://download.spotify.com/debian/pubkey_5384CE82BA52C83A.gpg",
)
# Configure aur to use builduser and the GNUPGHOME.
os.environ["GNUPGHOME"] = "/var/lib/builduser/gnupg"
decman.aur.makepkg_user = "builduser"
# Add version control systems required by the packages
decman.pacman.packages |= {"fossil"}
# Add AUR packages that require PGP keys or builduser setup
decman.aur.packages |= {"spotify", "pikchr-fossil"}
decman.modules += [um, gpg]
```
## Keys used in the decman store
- `aur_packages_for_module`
- `custom_packages_for_module`
## Configuration
This module has partially the same configuration with pacman. You'll have to define pacman output keywords and database options again.
```py
import decman
# set keywords
decman.aur.keywords = {"pacsave", "pacnew", "warning"}
# disable the feature
decman.aur.print_highlights = False
# signature level for querying existing databases
decman.aur.database_signature_level = 2048 # pyalpm.SIG_DATABASE_OPTIONAL
# path to databases
decman.aur.database_path = "/var/lib/pacman/"
```
There are some options related to building packages.
```py
# Timeout for fetching information from AUR
decman.aur.aur_rpc_timeout = 30
# User which builds AUR packages
decman.aur.makepkg_user = "nobody"
# Directory used for building packages
decman.aur.build_dir = "/tmp/decman/build"
```
Some AUR packages must be verified with GPG keys. In that case set the `GNUPGHOME` environment variable to the keystore containing imported keys. Set `makepkg_user` user to the owner of that directory.
```py
import os
os.environ["GNUPGHOME"] = "/home/kk/.gnupg/"
decman.aur.makepkg_user = "kk"
```
Additionally it's possible to override the commands this plugin uses. Create your own `AurCommands` class and override methods returning commands. Since this plugin and the pacman plugin have many overlapping commands, `AurCommands` is actually a subclass of `PacmanCommands`. This means that you can use a single override class for both of them. These are the defaults.
```py
from decman.plugins import aur
import decman
class MyAurAndPacmanCommands(aur.AurCommands):
def install_as_dependencies(self, pkgs: set[str]) -> list[str]:
"""
Running this command installs the given packages from pacman repositories.
The packages are installed as dependencies.
"""
return ["pacman", "-S", "--needed", "--asdeps"] + list(pkgs)
def install_files_as_dependencies(self, pkg_files: list[str]) -> list[str]:
"""
Running this command installs the given packages files as dependencies.
"""
return ["pacman", "-U", "--asdeps"] + pkg_files
def compare_versions(self, installed_version: str, new_version: str) -> list[str]:
"""
Running this command outputs -1 when the installed version is older than the new version.
"""
return ["vercmp", installed_version, new_version]
def git_clone(self, repo: str, dest: str) -> list[str]:
"""
Running this command clones a git repository to the the given destination.
"""
return ["git", "clone", repo, dest]
def git_diff(self, from_commit: str) -> list[str]:
"""
Running this command outputs the difference between the given commit and
the current state of the repository.
"""
return ["git", "diff", from_commit]
def git_get_commit_id(self) -> list[str]:
"""
Running this command outputs the current commit id.
"""
return ["git", "rev-parse", "HEAD"]
def git_log_commit_ids(self) -> list[str]:
"""
Running this command outputs commit hashes of the repository.
"""
return ["git", "log", "--format=format:%H"]
def review_file(self, file: str) -> list[str]:
"""
Running this command outputs a file for the user to see.
"""
return ["less", file]
def make_chroot(self, chroot_dir: str, with_pkgs: set[str]) -> list[str]:
"""
Running this command creates a new arch chroot to the chroot directory and installs the
given packages there.
"""
return ["mkarchroot", chroot_dir] + list(with_pkgs)
def install_chroot(self, chroot_dir: str, packages: list[str]):
"""
Running this command installs the given packages to the given chroot.
"""
return [
"arch-nspawn",
chroot_dir,
"pacman",
"-S",
"--needed",
"--noconfirm",
] + packages
def resolve_real_name_chroot(self, chroot_dir: str, pkg: str) -> list[str]:
"""
This command prints a real name of a package.
For example, it prints the package which provides a virtual package.
"""
return [
"arch-nspawn",
chroot_dir,
"pacman",
"-Sddp",
"--print-format=%n",
pkg,
]
def remove_chroot(self, chroot_dir: str, packages: set[str]):
"""
Running this command removes the given packages from the given chroot.
"""
return ["arch-nspawn", chroot_dir, "pacman", "-Rsu", "--noconfirm"] + list(packages)
def make_chroot_pkg(
self, chroot_wd_dir: str, user: str, pkgfiles_to_install: list[str]
) -> list[str]:
"""
Running this command creates a package file using the given chroot.
The package is created as the user and the pkg_files_to_install are installed
in the chroot before the package is created.
"""
makechrootpkg_cmd = ["makechrootpkg", "-c", "-r", chroot_wd_dir, "-U", user]
for pkgfile in pkgfiles_to_install:
makechrootpkg_cmd += ["-I", pkgfile]
return makechrootpkg_cmd
def print_srcinfo(self) -> list[str]:
"""
Running this command prints SRCINFO generated from the package in the current
working directory.
"""
return ["makepkg", "--printsrcinfo"]
# -------------------------------------------
# Here I override some PacmanCommand methods.
# -------------------------------------------
def set_as_explicit(self, pkgs: set[str]) -> list[str]:
"""
Running this command sets the given as explicitly installed.
"""
return ["pacman", "-D", "--asexplicit"] + list(pkgs)
def set_as_dependencies(self, pkgs: set[str]) -> list[str]:
"""
Running this command sets the given packages as dependencies.
"""
return ["pacman", "-D", "--asdeps"] + list(pkgs)
```
Applying the commands is easy.
```py
import decman
decman.pacman.commands = MyAurAndPacmanCommands()
decman.aur.commands = MyAurAndPacmanCommands()
```
+260
View File
@@ -0,0 +1,260 @@
# Extras
Decman ships with some built in modules. They implement functionality that is probably useful for declarative management, but for one reason or another don't make sense as plugins.
## User and group management module
```python
import decman.extras.users
```
A decman module for managing system users, groups, and supplementary group membership and subordinate UID/GID ranges for existing users.
The module is **additive**: it only manages users/groups you explicitly register, and it only manages additional groups/subids you explicitly define. Anything created manually and not tracked by this module is left alone.
### Provided types
#### `Group`
Represents a managed group.
```python
@dataclass(frozen=True)
class Group:
groupname: str
gid: Optional[int] = None
system: bool = False
```
Fields:
- `groupname`: Group name.
- `gid`: Desired numeric GID. If omitted, system assigns one.
- `system`: Only affects _creation_ (`groupadd --system`). Changing this after creation does nothing.
#### `User`
Represents a managed user.
```python
@dataclass(frozen=True)
class User:
username: str
uid: Optional[int] = None
group: Optional[str] = None
home: Optional[str] = None
shell: Optional[str] = None
groups: tuple[str, ...] = ()
system: bool = False
```
Fields:
- `username`: Login name.
- `uid`: Desired numeric UID. If omitted, system assigns one.
- `group`: Primary group name.
- `home`: Home directory.
- `shell`: Login shell.
- `groups`: Supplementary groups set.
- `system`: Only affects _creation_ (`useradd --system`). Changing this after creation does nothing.
### `UserManager` module
```python
class UserManager(Module):
```
#### Lifecycle
- Before update
- Create/modify managed groups.
- Create/modify managed users.
- Delete previously-managed users/groups that are no longer listed.
- After update
- Apply **additional** supplementary group membership and **subuid/subgid** ranges (including removals).
#### Store keys
The module persists state in decman store under these keys:
- `usermanager_users`
- `usermanager_groups`
- `usermanager_user_additional_groups`
- `usermanager_user_subuids`
- `usermanager_user_subgids`
The module does **not** parse `/etc/subuid` or `/etc/subgid`; it relies on these store keys to compute additions/removals.
#### Methods
##### `add_user(user: User)`
Ensure a user exists with the configured attributes.
Notes:
- If `uid` is provided and an existing user matches by UID but has a different name, the module will rename the user (`usermod --login`) and apply other changes.
##### `add_group(group: Group)`
Ensure a group exists with the configured attributes.
##### `add_user_to_group(user: str, group: str)`
Ensure `user` is a member of `group`.
- This is applied in `after_update`.
- Both `user` and `group` are expected to exist
You should not use this method for users added with `add_user`.
##### `add_subuids(user: str, first: int, last: int)`
Ensure subordinate UID range `first-last` is present for `user`.
##### `add_subgids(user: str, first: int, last: int)`
Ensure subordinate GID range `first-last` is present for `user`.
### Example usage
```python
from decman.extras.users import UserManager, User, Group
um = UserManager()
um.add_group(Group("containers", system=True))
um.add_user(User(
username="alice",
uid=1001,
group="users",
home="/home/alice",
groups=(),
shell="/bin/zsh",
))
um.add_user_to_group("bob", "containers")
um.add_subuids("alice", 100000, 165535)
um.add_subgids("alice", 100000, 165535)
import decman
decman.modules += [um]
```
## GPG receiver module
```python
import decman.extras.gpg
```
Manages importing OpenPGP public keys into per-user GnuPG homes. Tracks imported keys in the decman store and removes keys that were previously managed but are no longer configured.
This module is intentionally limited since it's main usage is for AUR build users. You probably shouldn't manage your primary users keyring with it.
### Types
#### `OwnerTrust`
Valid ownertrust levels:
- `never`
- `marginal`
- `full`
- `ultimate`
These map to GnuPG `--import-ownertrust` numeric levels `1..4`.
#### `SourceKind`
How a key is imported:
- `fingerprint`: fetch from keyserver via `--recv-keys`
- `uri`: fetch from URI via `--fetch-key`
- `file`: import from local file via `--import`
#### `Key`
Represents one managed key entry.
Fields:
- `fingerprint`: OpenPGP fingerprint, validated to be exactly 40 hex chars (spaces allowed in input; normalized by removing spaces and uppercasing).
- `source_kind`: one of `fingerprint | uri | file`.
- `source`: keyserver (for `fingerprint`), URI (for `uri`), or filepath (for `file`).
- `trust`: optional `OwnerTrust` to set via ownertrust import.
Validation behavior:
- Fingerprint is normalized: `replace(" ", "").upper()`.
- Fingerprint must match `^[0-9A-F]{40}$`; otherwise `ValueError`.
### `GPGReceiver` module
```python
class GPGReceiver(module.Module):
```
#### Store keys
The module persists state in decman store under these keys:
- `gpgreceiver_userhome_keys`
It relies on the store to keep track which keys were added by it.
#### Public API
##### `receive_key(user: str, gpg_home: str, fingerprint: str, keyserver: str, trust: OwnerTrust | None = None)`
Receives a key with a `fingerprint` from a `keyserver` to a `gpg_home` owned by `user`.
If `trust` is provided, ownertrust is set after import.
##### `fetch_key(user: str, gpg_home: str, fingerprint: str, uri: str, trust: OwnerTrust | None=None)`
Receives a key with a `fingerprint` from a `uri` to a `gpg_home` owned by `user`.
If `trust` is provided, ownertrust is set after import.
##### `import_key(user: str, gpg_home: str, fingerprint: str, file: str, trust: OwnerTrust | None =None)`
Receives a key with a `fingerprint` from a local `file` to a `gpg_home` owned by `user`.
If `trust` is provided, ownertrust is set after import.
### Example usage
```python
from decman.modules.gpg import GPGReceiver
import decman
gpg = GPGReceiver()
# Receive a key from a keyserver
gpg.receive_key(
user="builduser",
gpg_home="/var/lib/builduser/gnupg",
fingerprint="AAAA AAAA AAAA AAAA AAAA AAAA AAAA AAAA AAAA AAAA",
keyserver="hkps://keyserver.ubuntu.com",
trust="marginal",
)
# Fetch a key from a URI
gpg.fetch_key(
user="alice",
gpg_home="/home/alice/.gnupg",
fingerprint="BBBB BBBB BBBB BBBB BBBB BBBB BBBB BBBB BBBB BBBB",
uri="https://example.org/signing-key.asc",
)
# Import a key from a local file
gpg.import_key(
user="bob",
gpg_home="/home/bob/.gnupg",
fingerprint="CCCC CCCC CCCC CCCC CCCC CCCC CCCC CCCC CCCC CCCC",
file="/etc/decman/keys/custom.asc",
)
decman.modules += [gpg]
```
+136
View File
@@ -0,0 +1,136 @@
# Flatpak
The flatpak plugin is used to manage flatpak apps. It manages both systemd-wide and user-specific flatpaks. Flatpaks are still a new addition to decman, so they might not work as well as pacman packages. Flatpak management is disabled by default.
This plugin will ensure that installed flatpak apps match those defined in the decman source. If your system has installed a package, but it is not included in the source, it will be uninstalled. You don't need to list dependencies and runtimes in your source as those will be handeled by flatpak automatically. This plugin will remove unneeded runtimes.
## Usage
Define systemd-wide flatpaks.
```py
import decman
decman.flatpak.packages |= {"org.mozilla.firefox", "org.signal.Signal"}
```
Define user-specific flatpaks.
```py
decman.flatpak.user_packages.setdefault("user", {}).update({"com.valvesoftware.Steam"})
```
Define ignored flatpaks. This plugin won't install them nor remove them. This list affects user and system flatpaks.
```py
decman.flatpak.ignored_packages |= {"dev.zed.Zed"}
```
### Within modules
Modules can also define flatpaks units. Decorate a module's method with `@decman.plugins.flatpaks.packages` and return a `set[str]` of flatpak names from that module. For user flatpaks decorate with `@decman.plugins.flatpak.user_packages` and return a `dict[str, set[str]]` of usernames and flatpaks for that user.
```py
import decman
from decman.plugins import flatpak
class MyModule(decman.Module):
...
@flatpak.packages
def units_defined_in_this_module(self) -> set[str]:
return {"org.signal.Signal", "org.mozilla.firefox"}
@flatpak.user_packages
def user_units_defined_in_this_module(self) -> dict[str, set[str]]:
return {"user": {"com.valvesoftware.Steam"}}
```
If packages or user packages change, this plugin will flag the module as changed. The module's `on_change` method will be executed.
## Keys used in the decman store
- `flatpaks_for_module`
- `user_flatpaks_for_module`
## Configuration
It's possible to override the commands this plugin uses. Create your own `FlatpakCommands` class and override methods returning commands. These are the defaults.
```py
from decman.plugins import flatpak
class MyCommands(flatpak.FlatpakCommands):
def list_apps(self, as_user: bool) -> list[str]:
"""
Running this command outputs a newline separated list of installed flatpak application IDs.
If ``as_user`` is ``True``, run the command as the user whose packages should be listed.
NOTE: The first line says 'Application ID' and should be ignored.
"""
return [
"flatpak",
"list",
"--app",
"--user" if as_user else "--system",
"--columns",
"application",
]
def install(self, pkgs: set[str], as_user: bool) -> list[str]:
"""
Running this command installs all listed packages, and their dependencies/runtimes
automatically.
If ``as_user`` is ``True``, run the command as the user for whom packages are installed.
"""
return [
"flatpak",
"install",
"--user" if as_user else "--system",
] + sorted(pkgs)
def upgrade(self, as_user: bool) -> list[str]:
"""
Updates all installed flatpaks including runtimes and dependencies.
If ``as_user`` is ``True``, run the command as the user whose flatpaks are updated.
"""
return [
"flatpak",
"update",
"--user" if as_user else "--system",
]
def remove(self, pkgs: set[str], as_user: bool) -> list[str]:
"""
Running this command will remove the listed packages.
If ``as_user`` is ``True``, run the command as the user for whom packages are removed.
"""
return [
"flatpak",
"remove",
"--user" if as_user else "--system",
] + sorted(pkgs)
def remove_unused(self, as_user: bool) -> list[str]:
"""
This will remove all unused flatpak dependencies and runtimes.
If ``as_user`` is ``True``, run the command as the user for whom packages are removed.
"""
return [
"flatpak",
"remove",
"--unused",
"--user" if as_user else "--system",
]
```
Then set the commands.
```py
import decman
decman.flatpak.commands = MyCommands()
```
+765
View File
@@ -0,0 +1,765 @@
# Migrating to the new architecture
I recommend reading decman's new documentation. This document is supposed to be a quick reference on what will you have to modify in your current source to make it work with the new decman. This will not document new features.
## Few notes about changed behavior
This change is mostly architectural and doesn't change decman's behavior, but there are a few exceptions.
- The pacman plugin will now remove orphan packages.
- **Packages that are only optionally required by other packages are considered orphans.**
- Explicitly installed packages that are required by other explicitly installed packages are no longer uninstalled when removed from the source.
- Module's `on_disable` will now be executed when the module is no longer present in `decman.modules`.
- It will be executed even when the module is removed completely from the source
- Order of operations has changed. While the order of operations is now configurable, returning to the previous way is not possible.
- I will no longer provide any examples for using decman with other languages than Python. It would be possible to write an adapter, but I don't see it as worth the effort.
Since there are changes in the internal logic, it is possible that there are more breaking changes, but I haven't thought about them yet.
## After upgrading decman
After upgrading decman, the store and cache must be deleted. This will cause some `on_enable` -hooks to run again, but the store has had many internal changes, and should be recreated.
```sh
sudo rm /var/lib/decman/store.json
sudo rm -r /var/cache/decman/
```
## Changes
One notable change is replacing lists with sets. Sets make more sense for most things decman manages since duplicates and order are meaningless. With Python you'll want to use `|=` when adding two sets together instead of `+=` which is for lists.
### Files and directories
Files and directories are still managed the same way.
```py
import decman
decman.files["/etc/pacman.conf"] = File(source_file="./dotfiles/pacman.conf")
decman.directories["/home/user/.config/nvim"] = Directory(source_directory="./dotfiles/nvim",
```
### Pacman packages
#### Old
```py
decman.packages += ["devtools", "git", "networkmanager"]
decman.ignored_packages += ["rustup", "yay"]
```
#### New
Packages are now defined with the plugin. Sets are used instead of lists. Ignored packages contains only native packages found in the pacman repositories. Ignored AUR packages is a seperate setting.
```py
decman.pacman.packages |= {"devtools", "git", "networkmanager"}
decman.pacman.ignored_packages |= {"rustup"}
```
### AUR packages
#### Old
```py
decman.aur_packages += ["decman", "android-studio"]
decman.ignored_packages += ["rustup", "yay"]
```
#### New
Packages are now defined with the plugin. Sets are used instead of lists. Ignored packages contains only foreign packages for example AUR packages.
```py
decman.aur.packages |= {"decman", "android-studio"}
decman.aur.ignored_packages |= {"yay"}
```
### User Packages
#### Old
```py
decman.user_packages.append(
UserPackage(
pkgname="decman",
version="0.4.2",
provides=["decman"],
dependencies=[
"python",
"python-requests",
"devtools",
"pacman",
"systemd",
"git",
"less",
],
make_dependencies=[
"python-setuptools",
"python-build",
"python-installer",
"python-wheel",
],
git_url="https://github.com/kiviktnm/decman-pkgbuild.git",
)
)
```
#### New
User packages were renamed to custom packages. Sets are used instead of lists. PKGBUILDs are now parsed by decman so defining them is simpler. They are managed by the aur plugin.
```py
from decman.plugins import aur
decman.aur.custom_packages |= {aur.CustomPackage("decman", git_url="https://github.com/kiviktnm/decman-pkgbuild.git")}
```
### Systemd services
#### Old
```py
decman.enabled_systemd_units += ["NetworkManager.service"]
decman.enabled_systemd_user_units.setdefault("kk", []).append("syncthing.service")
```
#### New
Units are now defined with the plugin. Sets are used instead of lists.
```py
decman.systemd.enabled_units |= {"NetworkManager.service"}
decman.systemd.enabled_user_units.setdefault("user", set()).add("syncthing.service")
```
### Flatpaks
#### Old
```py
decman.flatpak_packages += ["org.mozilla.firefox"]
decman.ignored_flatpak_packages += ["org.signal.Signal"]
decman.flatpak_user_packages.setdefault("kk", []).append("com.valvesoftware.Steam")
```
#### New
Packages are now defined with the plugin. Sets are used instead of lists.
```py
decman.flatpak.packages |= {"org.mozilla.firefox"}
decman.flatpak.ignored_packages |= {"org.signal.Signal"}
decman.flatpak.user_packages.setdefault("kk", {}).update({"com.valvesoftware.Steam"})
```
### Changes to modules
#### Old
```py
import decman
from decman import Module, prg, sh
decman.modules += [MyModule()]
class MyModule(Module):
def __init__(self):
self.pkgs = ["rust"]
self.update_rustup = False
super().__init__(name="Example module", enabled=True, version="1")
def enable_my_custom_feature(self, b: bool):
if b:
self.pkgs = ["rustup"]
self.update_rustup = True
def on_enable(self):
sh("groupadd mygroup")
prg(["usermod", "--append", "--groups", "mygroup", "kk"])
def on_disable(self):
sh("whoami", user="kk")
sh("echo $HI", env_overrides={"HI": "Hello!"})
def after_update(self):
if self.update_rustup:
prg(["rustup", "update"], user="kk")
def after_version_change(self):
prg(["mkinitcpio", "-P"])
def file_variables(self) -> dict[str, str]:
return {"%msg%": "Hello, world!"}
def files(self) -> dict[str, File]:
return {
"/usr/local/bin/say-hello": File(
content="#!/usr/bin/env bash\necho %msg%", permissions=0o755
),
"/usr/local/share/say-hello/image.png": File(
source_file="files/i-dont-exist.png", bin_file=True
),
}
def directories(self) -> dict[str, Directory]:
return {
"/home/kk/.config/mod-app/": Directory(
source_directory="files/app-config", owner="kk"
)
}
def pacman_packages(self) -> list[str]:
return self.pkgs
def user_packages(self) -> list[UserPackage]:
return [UserPackage(...)]
def aur_packages(self) -> list[str]:
return ["protonvpn"]
def flatpak_packages(self) -> list[str]:
return ["org.mozilla.firefox"]
def flatpak_user_packages(self) -> dict[str, list[str]]:
return {"username": ["io.github.kolunmi.Bazaar"]}
def systemd_units(self) -> list[str]:
return ["reflector.timer"]
def systemd_user_units(self) -> dict[str, list[str]]:
return {"kk": ["syncthing.service"]}
```
#### New
Modules no longer have `version`s or `enabled` values. A module is enabled when it gets added to `decman.modules` and disabled when it gets removed from `decman.modules`. Versions are no longer needed because `after_version_change` has been removed and `on_change` has been added. `on_change` is executed automatically after the content of the module changes. `on_disable` will be executed automatically when the module is removed from `decman.modules`. It is no longer a instance method. Instead it must be a self-contained method with no references outside it. Not even imports.
Module methods will get a `Store` instance passed to them as an argument. It can be used to store key-value pairs between decman runs.
Files and directories work the same way as before. Pacman, aur and flatpak packages as well as systemd units have been changed. You'll no longer override methods on the `Module`-class. Instead you'll decorate any method with the appropriate decorator and return desired values from that method.
```py
import decman
from decman import Module, Store, prg, sh
from decman.plugins import pacman, aur, systemd, flatpak
decman.modules += [MyModule()]
class MyModule(Module):
def __init__(self):
self.pkgs = {"rust"}
self.update_rustup = False
super().__init__("Example module")
def enable_my_custom_feature(self, b: bool):
if b:
self.pkgs = {"rustup"}
self.update_rustup = True
def on_enable(self, store: Store):
sh("groupadd mygroup")
prg(["usermod", "--append", "--groups", "mygroup", "kk"])
store["value"] = True
@staticmethod
def on_disable():
from decman import sh
sh("whoami", user="kk")
sh("echo $HI", env_overrides={"HI": "Hello!"})
def after_update(self, store: Store):
if self.update_rustup:
prg(["rustup", "update"], user="kk")
def on_change(self, store: Store):
prg(["mkinitcpio", "-P"])
def file_variables(self) -> dict[str, str]:
return {"%msg%": "Hello, world!"}
def files(self) -> dict[str, File]:
return {
"/usr/local/bin/say-hello": File(
content="#!/usr/bin/env bash\necho %msg%", permissions=0o755
),
"/usr/local/share/say-hello/image.png": File(
source_file="files/i-dont-exist.png", bin_file=True
),
}
def directories(self) -> dict[str, Directory]:
return {
"/home/kk/.config/mod-app/": Directory(
source_directory="files/app-config", owner="kk"
)
}
@pacman.packages
def my_pacman_packages(self) -> set[str]:
return self.pkgs
@aur.custom_packages
def my_user_packages(self) -> set[aur.CustomPackage]:
return [aur.CustomPackage(...)]
@aur.packages
def my_aur_packages(self) -> set[str]:
return {"protonvpn-cli"}
@flatpak.packages
def my_flatpak_packages(self) -> set[str]:
return {"org.mozilla.firefox"}
@flatpak.user_packages
def my_flatpak_user_packages(self) -> dict[str, set[str]]:
return {"username": {"io.github.kolunmi.Bazaar"}}
@systemd.units
def my_systemd_units(self) -> set[str]:
return {"reflector.timer"}
@systemd.user_units
def my_systemd_user_units(self) -> dict[str, set[str]]:
return {"kk": {"syncthing.service"}}
```
## Configuration changes
With the plugin architecture, plugins now contain their own configuration instead of a global `decman.config`. The global `decman.config` still exists but the options available there are much more limited.
### Global options
#### Old
```py
import decman.config
decman.config.debug_output = False
decman.config.suppress_command_output = True
decman.config.quiet_output = False
```
#### New
`suppress_command_output` got removed. Commands that this option affected will now print their output only when encountering errors. In future releases the debug output option will be used to make it available even when not encountering errors.
Other options stayed the same. These will now override values passed as CLI arguments.
```py
decman.config.debug_output = False
decman.config.quiet_output = False
```
### Seperately enabled features
#### Old
```py
decman.config.enable_fpm = True
decman.config.enable_flatpak = False
```
#### New
These options are managed by setting `decman.execution_order`. Add or remove steps as needed.
```py
import decman
decman.execution_order = [
"files",
"pacman",
"aur", # AUR/fpm enabled
"systemd",
# "flatpak", # Flatpak disabled
]
```
### Pacman options
#### Old
```py
decman.config.pacman_output_keywords = [
"pacsave",
"pacnew",
]
decman.config.print_pacman_output_highlights = True
```
#### New
These are now moved under the pacman plugin and renamed. Keywords is no longer a `list`. It is now a `set`.
```py
import decman
decman.pacman.keywords = {"pacsave", "pacnew"}
decman.pacman.print_highlights = True
```
You'll have to set them for the aur plugin seperately. I recommend sharing the values between the plugins.
```py
import decman
decman.aur.keywords = {"pacsave", "pacnew"}
decman.aur.print_highlights = False
```
### Foreign package management related options
#### Old
```py
decman.config.aur_rpc_timeout = 30
decman.config.makepkg_user = "kk"
decman.config.build_dir = "/tmp/decman/build"
decman.config.pkg_cache_dir = "/var/cache/decman"
decman.config.number_of_packages_stored_in_cache = 3
decman.config.valid_pkgexts = [
".pkg.tar",
".pkg.tar.gz",
".pkg.tar.bz2",
".pkg.tar.xz",
".pkg.tar.zst",
".pkg.tar.lzo",
".pkg.tar.lrz",
".pkg.tar.lz4",
".pkg.tar.lz",
".pkg.tar.Z",
]
```
#### New
Options `number_of_packages_stored_in_cache` and `valid_pkgexts` got removed. The default values are no longer configurable. I deemed these settings unnecessary.
`pkg_cache_dir` is now a global setting and is used more generally for all cached things. Package cache is the directory `aur/` in this directory.
```py
decman.config.cache_dir = "/var/cache/decman"
```
Other options are now moved under the aur plugin.
```py
decman.aur.aur_rpc_timeout = 30
decman.aur.makepkg_user = "nobody"
decman.aur.build_dir = "/tmp/decman/build"
```
### Commands
Command management has now also been split up. Instead of a single commands class. Commands have to be overridden seperately for each plugin (except for AUR and pacman).
#### Old
Here are the old defaults.
```py
decman.config.commands = MyCommands()
class MyCommands(decman.config.Commands):
def list_pkgs(self) -> list[str]:
return ["pacman", "-Qeq", "--color=never"]
def list_flatpak_pkgs(self, as_user: bool = False) -> list[str]:
return [
"flatpak",
"list",
"--app",
"--user" if as_user else "--system",
"--columns",
"application",
]
def list_foreign_pkgs_versioned(self) -> list[str]:
return ["pacman", "-Qm", "--color=never"]
def install_pkgs(self, pkgs: list[str]) -> list[str]:
return ["pacman", "-S", "--color=always", "--needed"] + pkgs
def install_flatpak_pkgs(self, pkgs: list[str], as_user: bool = False) -> list[str]:
return ["flatpak", "install", "-y", "--user" if as_user else "--system"] + pkgs
def install_files(self, pkg_files: list[str]) -> list[str]:
return ["pacman", "-U", "--color=always", "--asdeps"] + pkg_files
def set_as_explicitly_installed(self, pkgs: list[str]) -> list[str]:
return ["pacman", "-D", "--color=always", "--asexplicit"] + pkgs
def install_deps(self, deps: list[str]) -> list[str]:
return ["pacman", "-S", "--color=always", "--needed", "--asdeps"] + deps
def is_installable(self, pkg: str) -> list[str]:
return ["pacman", "-Sddp", pkg]
def upgrade(self) -> list[str]:
return ["pacman", "-Syu", "--color=always"]
def upgrade_flatpak(self, as_user: bool = False) -> list[str]:
return [
"flatpak",
"update",
"--noninteractive",
"-y",
"--user" if as_user else "--system",
]
def remove(self, pkgs: list[str]) -> list[str]:
return ["pacman", "-Rs", "--color=always"] + pkgs
def remove_flatpak(self, pkgs: list[str], as_user: bool = False) -> list[str]:
return [
"flatpak",
"remove",
"--noninteractive",
"-y",
"--user" if as_user else "--system",
] + pkgs
def remove_unused_flatpak(self, as_user: bool = False) -> list[str]:
return [
"flatpak",
"remove",
"--noninteractive",
"-y",
"--unused",
"--user" if as_user else "--system",
]
def enable_units(self, units: list[str]) -> list[str]:
return ["systemctl", "enable"] + units
def disable_units(self, units: list[str]) -> list[str]:
return ["systemctl", "disable"] + units
def enable_user_units(self, units: list[str], user: str) -> list[str]:
return ["systemctl", "--user", "-M", f"{user}@", "enable"] + units
def disable_user_units(self, units: list[str], user: str) -> list[str]:
return ["systemctl", "--user", "-M", f"{user}@", "disable"] + units
def compare_versions(self, installed_version: str, new_version: str) -> list[str]:
return ["vercmp", installed_version, new_version]
def git_clone(self, repo: str, dest: str) -> list[str]:
return ["git", "clone", repo, dest]
def git_diff(self, from_commit: str) -> list[str]:
return ["git", "diff", from_commit]
def git_get_commit_id(self) -> list[str]:
return ["git", "rev-parse", "HEAD"]
def git_log_commit_ids(self) -> list[str]:
return ["git", "log", "--format=format:%H"]
def review_file(self, file: str) -> list[str]:
return ["less", file]
def make_chroot(self, chroot_dir: str, with_pkgs: list[str]) -> list[str]:
return ["mkarchroot", chroot_dir] + with_pkgs
def install_chroot_packages(self, chroot_dir: str, packages: list[str]):
return [
"arch-nspawn",
chroot_dir,
"pacman",
"-S",
"--needed",
"--noconfirm",
] + packages
def resolve_real_name(self, chroot_dir: str, pkg: str) -> list[str]:
return [
"arch-nspawn",
chroot_dir,
"pacman",
"-Sddp",
"--print-format=%n",
pkg,
]
def remove_chroot_packages(self, chroot_dir: str, packages: list[str]):
return ["arch-nspawn", chroot_dir, "pacman", "-Rsu", "--noconfirm"] + packages
def make_chroot_pkg(
self, chroot_wd_dir: str, user: str, pkgfiles_to_install: list[str]
) -> list[str]:
makechrootpkg_cmd = ["makechrootpkg", "-c", "-r", chroot_wd_dir, "-U", user]
for pkgfile in pkgfiles_to_install:
makechrootpkg_cmd += ["-I", pkgfile]
return makechrootpkg_cmd
```
#### New
AUR and pacman commands are a seperate setting, but they share the same subclass, so it's possible to set them in a one place. Many pacman query commands have been deleted since pyalpm is used now. New commands have also been added but it is better to look at the plugin documentation for those options.
These values are the new defaults.
```py
import decman
from decman.plugins import aur
decman.aur.commands = MyAurAndPacmanCommands()
decman.pacman.commands = MyAurAndPacmanCommands()
class MyAurAndPacmanCommands(aur.AurCommands):
def install(self, pkgs: set[str]) -> list[str]:
return ["pacman", "-S", "--needed"] + list(pkgs)
def upgrade(self) -> list[str]:
return ["pacman", "-Syu"]
def set_as_dependencies(self, pkgs: set[str]) -> list[str]:
return ["pacman", "-D", "--asdeps"] + list(pkgs)
def set_as_explicit(self, pkgs: set[str]) -> list[str]:
return ["pacman", "-D", "--asexplicit"] + list(pkgs)
def remove(self, pkgs: set[str]) -> list[str]:
return ["pacman", "-Rs"] + list(pkgs)
def install_as_dependencies(self, pkgs: set[str]) -> list[str]:
return ["pacman", "-S", "--needed", "--asdeps"] + list(pkgs)
def install_files_as_dependencies(self, pkg_files: list[str]) -> list[str]:
return ["pacman", "-U", "--asdeps"] + pkg_files
def compare_versions(self, installed_version: str, new_version: str) -> list[str]:
return ["vercmp", installed_version, new_version]
def git_clone(self, repo: str, dest: str) -> list[str]:
return ["git", "clone", repo, dest]
def git_diff(self, from_commit: str) -> list[str]:
return ["git", "diff", from_commit]
def git_get_commit_id(self) -> list[str]:
return ["git", "rev-parse", "HEAD"]
def git_log_commit_ids(self) -> list[str]:
return ["git", "log", "--format=format:%H"]
def review_file(self, file: str) -> list[str]:
return ["less", file]
def make_chroot(self, chroot_dir: str, with_pkgs: set[str]) -> list[str]:
return ["mkarchroot", chroot_dir] + list(with_pkgs)
def install_chroot(self, chroot_dir: str, packages: list[str]):
return [
"arch-nspawn",
chroot_dir,
"pacman",
"-S",
"--needed",
"--noconfirm",
] + packages
def resolve_real_name_chroot(self, chroot_dir: str, pkg: str) -> list[str]:
return [
"arch-nspawn",
chroot_dir,
"pacman",
"-Sddp",
"--print-format=%n",
pkg,
]
def remove_chroot(self, chroot_dir: str, packages: set[str]):
return ["arch-nspawn", chroot_dir, "pacman", "-Rsu", "--noconfirm"] + list(packages)
def make_chroot_pkg(
self, chroot_wd_dir: str, user: str, pkgfiles_to_install: list[str]
) -> list[str]:
makechrootpkg_cmd = ["makechrootpkg", "-c", "-r", chroot_wd_dir, "-U", user]
for pkgfile in pkgfiles_to_install:
makechrootpkg_cmd += ["-I", pkgfile]
return makechrootpkg_cmd
def print_srcinfo(self) -> list[str]:
return ["makepkg", "--printsrcinfo"]
```
Systemd commands:
```py
import decman
from decman.plugins import systemd
decman.systemd.commands = MyCommands()
class MyCommands(SystemdCommands):
def enable_units(self, units: set[str]) -> list[str]:
return ["systemctl", "enable"] + list(units)
def disable_units(self, units: set[str]) -> list[str]:
return ["systemctl", "disable"] + list(units)
def enable_user_units(self, units: set[str], user: str) -> list[str]:
return ["systemctl", "--user", "-M", f"{user}@", "enable"] + list(units)
def disable_user_units(self, units: set[str], user: str) -> list[str]:
return ["systemctl", "--user", "-M", f"{user}@", "disable"] + list(units)
def daemon_reload(self) -> list[str]:
return ["systemctl", "daemon-reload"]
def user_daemon_reload(self, user: str) -> list[str]:
return ["systemctl", "--user", "-M", f"{user}@", "daemon-reload"]
```
Flatpak commands:
```py
import decman
from decman.plugins import flatpak
decman.flatpak.commands = MyCommands()
class MyCommands(FlatpakCommands):
def list_apps(self, as_user: bool) -> list[str]:
return [
"flatpak",
"list",
"--app",
"--user" if as_user else "--system",
"--columns",
"application",
]
def install(self, pkgs: set[str], as_user: bool) -> list[str]:
return [
"flatpak",
"install",
"--user" if as_user else "--system",
] + sorted(pkgs)
def upgrade(self, as_user: bool) -> list[str]:
return [
"flatpak",
"update",
"--user" if as_user else "--system",
]
def remove(self, pkgs: set[str], as_user: bool) -> list[str]:
return [
"flatpak",
"remove",
"--user" if as_user else "--system",
] + sorted(pkgs)
def remove_unused(self, as_user: bool) -> list[str]:
return [
"flatpak",
"remove",
"--unused",
"--user" if as_user else "--system",
]
```
+118
View File
@@ -0,0 +1,118 @@
# Pacman
Pacman plugin can be used to manage pacman packages. The pacman plugin manages only native packages found in arch repositories. All foreign (AUR) packages are ignored by this plugin.
This plugin will ensure that explicitly installed packages match those defined in the decman source. If your system has explicitly installed package A, but it is not included in the source, it will be uninstalled. You don't need to list dependencies in your source as those will be handeled by pacman automatically. However, if you have inluded package B in your source and that package depends on A, this plugin will not remove A. Instead it will demote A to a dependency. This plugin will also remove all orphaned packages automatically. **Packages that are only optionally required by other packages are considered orphans.** This way this plugin can ensure that your system truly matches your source. You cannot install an optional dependency, and forget about it later.
Please keep in mind that decman doesn't play well with package groups, since all packages part of that group will be installed explicitly. After the initial run decman will now try to remove those packages since it only knows that the group itself should be explicitly installed. Instead of package groups, use meta packages.
## Usage
Define system packages.
```py
import decman
decman.pacman.packages |= {"sudo", "vim"}
```
Define ignored packages. This plugin won't install them nor remove them.
```py
# Include only packages found in the pacman repositories in here.
decman.pacman.ignored_packages |= {"opendoas"}
```
This plugin's execution order step name is `pacman`.
### Within modules
Modules can also define pacman packages. Decorate a module's method with `@decman.plugins.pacman.packages` and return a `set[str]` of package names from that module.
```py
import decman
from decman.plugins import pacman
class MyModule(decman.Module):
...
@pacman.packages
def packages_defined_in_this_module(self) -> set[str]:
return {"tmux", "kitty"}
```
If this set changes, this plugin will flag the module as changed. The module's `on_change` method will be executed.
## Keys used in the decman store
- `packages_for_module`
## Configuration
This plugin has a pacman output highlight function. If pacman output contains some keywords, it will be highlighted. You can disable this feature or set the keywords.
```py
import decman
# set keywords
decman.pacman.keywords = {"pacsave", "pacnew", "warning"}
# disable the feature
decman.pacman.print_highlights = False
# signature level for querying existing databases
decman.pacman.database_signature_level = 2048 # pyalpm.SIG_DATABASE_OPTIONAL
# path to databases
decman.pacman.database_path = "/var/lib/pacman/"
```
Additionally it's possible to override the commands this plugin uses. Create your own `PacmanCommands` class and override methods returning commands. These are the defaults.
```py
from decman.plugins import pacman
class MyCommands(pacman.PacmanCommands):
def list_pacman_repos(self) -> list[str]:
"""
Running this command prints a newline seperated list of pacman repositories.
"""
return ["pacman-conf", "--repo-list"]
def install(self, pkgs: set[str]) -> list[str]:
"""
Running this command installs the given packages from pacman repositories.
"""
return ["pacman", "-S", "--needed"] + list(pkgs)
def upgrade(self) -> list[str]:
"""
Running this command upgrades all pacman packages from pacman repositories.
"""
return ["pacman", "-Syu"]
def set_as_dependencies(self, pkgs: set[str]) -> list[str]:
"""
Running this command sets the given packages as dependencies.
"""
return ["pacman", "-D", "--asdeps"] + list(pkgs)
def set_as_explicit(self, pkgs: set[str]) -> list[str]:
"""
Running this command sets the given as explicitly installed.
"""
return ["pacman", "-D", "--asexplicit"] + list(pkgs)
def remove(self, pkgs: set[str]) -> list[str]:
"""
Running this command removes the given packages and their dependencies
(that aren't required by other packages).
"""
return ["pacman", "-Rs"] + list(pkgs)
```
Then set the commands.
```py
import decman
decman.pacman.commands = MyCommands()
```
+107
View File
@@ -0,0 +1,107 @@
# Systemd
The systemd plugin can enable systemd services, system wide or for a specific user. It will enable all units defined in the source, and disable them when they are removed from the source.
This plugin manages systemd units "softly". It only touches units included in the source. So if you install a package that automatically enables a systemd unit, you don't have to include it in the source. If a unit is not defined in the source, the plugin will not touch it.
Decman will only enable and disable systemd services. It will not start or stop them. Starting or stopping them automatically can cause issues.
## Usage
Declare system-wide units.
```py
import decman
decman.systemd.enabled_units |= {"NetworkManager.service", "ufw.service"}
```
Declare user units. Here this `setdefault` method is used to ensure that the key `user` exists. In this case you cannot use the `|=` syntax and instead must call the `update`-method.
```py
decman.systemd.enabled_user_units.setdefault("user", set()).update({"syncthing.service"})
```
This plugin's execution order step name is `systemd`.
### Within modules
Modules can also define systemd units. Decorate a module's method with `@decman.plugins.systemd.units` and return a `set[str]` of package names from that module. For user units decorate with `@decman.plugins.systemd.user_units` and return a `dict[str, set[str]]` of usernames and user units.
```py
import decman
from decman.plugins import systemd
class MyModule(decman.Module):
...
@systemd.units
def units_defined_in_this_module(self) -> set[str]:
return {"NetworkManager.service", "ufw.service"}
@systemd.user_units
def user_units_defined_in_this_module(self) -> dict[str, set[str]]:
return {"user": {"syncthing.service"}}
```
If units or user units change, this plugin will flag the module as changed. The module's `on_change` method will be executed.
## Keys used in the decman store
- `systemd_units_for_module`
- `systemd_user_units_for_module`
## Configuration
It's possible to override the commands this plugin uses. Create your own `SystemdCommands` class and override methods returning commands. These are the defaults.
```py
from decman.plugins import systemd
class MyCommands(systemd.SystemdCommands):
"""
Default commands for the Systemd plugin.
"""
def enable_units(self, units: set[str]) -> list[str]:
"""
Running this command enables the given systemd units.
"""
return ["systemctl", "enable"] + list(units)
def disable_units(self, units: set[str]) -> list[str]:
"""
Running this command disables the given systemd units.
"""
return ["systemctl", "disable"] + list(units)
def enable_user_units(self, units: set[str], user: str) -> list[str]:
"""
Running this command enables the given systemd units for the user.
"""
return ["systemctl", "--user", "-M", f"{user}@", "enable"] + list(units)
def disable_user_units(self, units: set[str], user: str) -> list[str]:
"""
Running this command disables the given systemd units for the user.
"""
return ["systemctl", "--user", "-M", f"{user}@", "disable"] + list(units)
def daemon_reload(self) -> list[str]:
"""
Running this command reloads the systemd daemon.
"""
return ["systemctl", "daemon-reload"]
def user_daemon_reload(self, user: str) -> list[str]:
"""
Running this command reloads the systemd daemon for the given user.
"""
return ["systemctl", "--user", "-M", f"{user}@", "daemon-reload"]
```
Then set the commands.
```py
import decman
decman.systemd.commands = MyCommands()
```
+212
View File
@@ -0,0 +1,212 @@
# Example
This directory contains an example of a minimal decman configuration. This also functions as a tutorial for starting out with decman. I recommend looking at the [docs](/docs/README.md) after this.
## Tutorial
### Installing decman
I will first install git and base-devel. Then I'll clone the PKGBUILD and install decman.
```sh
sudo pacman -S git base-devel
git clone https://aur.archlinux.org/decman.git
cd decman/
makepkg -sic
```
### Starting out
I will create a source directory for the system's configuration.
```sh
mkdir ~/source
cd ~/source
```
Decman will remove all explicitly installed packages not found in the source. Let's find all explicitly installed packages.
```sh
$ pacman -Qeq
base
base-devel
btrfs-progs
decman
dosfstools
efibootmgr
git
grub
linux
openssh
qemu-guest-agent
sudo
vim
```
First thing to note: `decman` is not a native package. I remember this, but if you don't, you can find only native packages with `pacman -Qeqn` and foreign packages with `pacman -Qeqm`. Since decman is not a native package, the pacman plugin cannot handle it. I'll add decman to AUR packages.
Instead of adding all of these packages to `decman.pacman.packages`, I will first create a module for base system packages in `~/source/base.py`.
```py
import decman
from decman.plugins import pacman, aur
class BaseModule(decman.Module):
def __init__(self):
# I'll intend this module to be a singleton (only one instance ever),
# so I'll inline the module name
super().__init__("base")
@pacman.packages
def pkgs(self) -> set[str]:
return {
"base",
"btrfs-progs",
"dosfstools",
"efibootmgr",
"grub",
"linux",
# I'll also include git and base-devel here, they are essential to this system
"git",
"base-devel",
}
@aur.packages
def aurpkgs(self) -> set[str]:
return {"decman"}
```
Then I'll create the main source file with the rest of the packages. I'll import `BaseModule` and add it to `decman.modules`. The main file is `~/source/source.py`.
```py
import decman
from base import BaseModule
decman.pacman.packages |= {"openssh", "qemu-guest-agent", "sudo", "vim"}
decman.modules += [BaseModule()]
```
This config is already enough to run decman for the first time.
```sh
sudo decman --source /home/arch/source/source.py
```
This will run a system upgrade, but otherwise nothing else happens, since my system already matches the desired configuration.
### Extending my config with files and commands
Now I'll want to gradually add more stuff to my config. As an example, I'll add my custom `mkinitcpio.conf`. I'll create the file `~/source/files/mkinitcpio.conf` with the desired content. Then I'll add the file to my `BaseModule`. Since I want to run the command `mkinitcpio -P` every time I update my config, I'll add a on change hook as well. I'll update the file `~/source/base.py`.
```py
class BaseModule(decman.Module):
...
def files(self) -> dict[str, decman.File]:
return {"/etc/mkinitcpio.conf": decman.File(source_file="./files/mkinitcpio.conf")}
def on_change(self, store):
decman.prg(["mkinitcpio", "-P"])
```
I'll also add my vim config to decman. I could now create a Vim module, but since my config is simple, I feel that is not needed. I'll update the main source file `~/source/source.py`.
```py
import decman
...
decman.files["/home/arch/.vimrc"] = decman.File(source_file="./files/vimrc", owner="arch", permissions=0o600)
```
Then I'll apply my changes. Decman will remember my source, so no need to give it as an argument anymore. I don't want to waste time checking for aur updates, so I'll skip them.
```sh
sudo decman --skip aur
```
### Systemd services and flatpaks
I want add a desktop environment. I'll create a module for that in the file `~/source/kde.py`. I'll use SDDM as the login manager. SDDM service needs to be enabled, so I'll use the systemd plugin for that.
```py
import decman
from decman.plugins import pacman, systemd
class KDE(decman.Module):
def __init__(self):
super().__init__("kde")
@pacman.packages
def pkgs(self) -> set[str]:
return {
"plasma-desktop",
"konsole",
"sddm",
}
@systemd.units
def units(self) -> set[str]:
return {"sddm.service"}
```
I'll add the module to enabled modules in `~/source/source.py`.
```py
import decman
from base import BaseModule
from kde import KDE
...
decman.modules += [BaseModule(), KDE()]
```
I'll run decman once again. I'll also start SDDM manually, since decman can't autostart it.
```sh
sudo decman
sudo systemctl start sddm
```
Lastly I want to install some packages with flatpak. I'll first have to install flatpak to make the plugin available. I'll do it manually since it's quicker.
```sh
sudo pacman -S flatpak
```
Then I'll modify `~/source/source.py`. I must add `flatpak` to execution steps to run the plugin.
```py
import decman
...
decman.execution_order = [
"files",
"pacman",
"aur",
"flatpak",
"systemd",
]
decman.pacman.packages.add("flatpak")
decman.flatpak.packages |= {"org.mozilla.firefox", "org.signal.Signal"}
```
Then run decman.
```sh
sudo decman
```
### Maintaining a system with decman
Decman is intended to replace your upgrade procedures. Instead of running `yay -Syu` for example, you would run `sudo decman`. With `after_update` hooks you can chain other update commands such as `rustup update`. This way you'll only have to remember to run decman. All other update steps are defined in your source.
## Plugins
It is possible to create your own plugins for decman. However, you probably won't need to do that, as modules are already very capable. This example directory also contains a **very** minimal plugin. To learn more about plugins, look at [the docs](/docs/README.md).
+33
View File
@@ -0,0 +1,33 @@
import decman
from decman.plugins import aur, pacman
class BaseModule(decman.Module):
def __init__(self):
# I'll intend this module to be a singleton (only one instance ever),
# so I'll inline the module name
super().__init__("base")
@pacman.packages
def pkgs(self) -> set[str]:
return {
"base",
"btrfs-progs",
"dosfstools",
"efibootmgr",
"grub",
"linux",
# I'll also include git and base-devel here, they are essential to this system
"git",
"base-devel",
}
@aur.packages
def aurpkgs(self) -> set[str]:
return {"decman"}
def files(self) -> dict[str, decman.File]:
return {"/etc/mkinitcpio.conf": decman.File(source_file="./files/mkinitcpio.conf")}
def on_change(self, store):
decman.prg(["mkinitcpio", "-P"])
-1
View File
@@ -1 +0,0 @@
# Imagine something here
-3
View File
@@ -1,3 +0,0 @@
Why are you looking here?
What is '%msg%'?
@@ -1 +0,0 @@
Thats right!
+4
View File
@@ -0,0 +1,4 @@
MODULES=()
BINARIES=()
FILES=()
HOOKS=(base systemd autodetect microcode modconf kms keyboard keymap sd-vconsole block filesystems fsck)
-2
View File
@@ -1,2 +0,0 @@
#!/usr/bin/env bash
echo "Hello, World!"
+2
View File
@@ -0,0 +1,2 @@
set number
syntax on
+19
View File
@@ -0,0 +1,19 @@
import decman
from decman.plugins import pacman, systemd
class KDE(decman.Module):
def __init__(self):
super().__init__("kde")
@pacman.packages
def pkgs(self) -> set[str]:
return {
"plasma-desktop",
"konsole",
"sddm",
}
@systemd.units
def units(self) -> set[str]:
return {"sddm.service"}
-113
View File
@@ -1,113 +0,0 @@
# from import is ok for importing classes and functions
# just remember to not import variables this way
from decman import Module, File, Directory, UserPackage, sh, prg
class MyModule(Module):
def __init__(self):
self.pkgs = ["rust"]
self.update_rustup = False
# Modules have names and versions.
# Names must be unique.
# If you disable a module, all packages, files etc assocated with module are removed.
super().__init__(name="Example module", enabled=True, version="1")
# You can add any methods etc to your modules.
def enable_my_custom_feature(self, b: bool):
if b:
self.pkgs = ["rustup"]
self.update_rustup = True
# This is ran, when the module gets enabled
def on_enable(self):
# Run arbitary shell code easily with the included sh function.
sh("groupadd mygroup")
# or run a program with arguments.
prg(["usermod", "--append", "--groups", "mygroup", "kk"])
def on_disable(self):
# You can run commands as any user
sh("whoami", user="kk")
# And override environment variables
sh("echo $HI", env_overrides={"HI": "Hello!"})
# Same options apply to prg as well.
def after_update(self):
# Run code after running decman.
if self.update_rustup:
prg(["rustup", "update"], user="kk")
def after_version_change(self):
# Modules have version numbers to allow conditionally running code.
# You could for example run mkinitcpio only after your config has changed.
# Just remember to change the version number.
prg(["mkinitcpio", "-P"])
# Files defined here are the same as outside of modules.
# There is however an additional feature:
# You may add variables to text files, that will be replaced with the given value.
def file_variables(self) -> dict[str, str]:
return {"%msg%": "Hello, world!"}
def files(self) -> dict[str, File]:
# Variables are substituted in text files automatically.
return {
"/usr/local/bin/say-hello":
File(content="#!/usr/bin/env bash\necho %msg%", permissions=0o755),
# Variables are not substituted in binary files.
"/usr/local/share/say-hello/image.png":
File(source_file="files/i-dont-exist.png", bin_file=True),
}
def directories(self) -> dict[str, Directory]:
# Directories are handeled the same way. Variables are substituted in text files.
return {
"/home/kk/.config/mod-app/":
Directory(source_directory="files/app-config", owner="kk")
}
# Packages and systemd units are basically the same with modules as without modules.
def pacman_packages(self) -> list[str]:
# Return pacman packages depending on the usage of this module.
return self.pkgs
def user_packages(self) -> list[UserPackage]:
return [
UserPackage(
pkgname="decman-git",
version="0.2.1",
provides=["decman"],
dependencies=[
"python",
"python-requests",
"devtools",
"systemd",
"pacman",
"git",
"less",
],
make_dependencies=[
"python-setuptools",
"python-build",
"python-installer",
"python-wheel",
],
git_url="https://github.com/kiviktnm/decman-pkgbuild.git",
)
]
def aur_packages(self) -> list[str]:
return ["protonvpn"]
def systemd_units(self) -> list[str]:
return ["reflector.timer"]
def systemd_user_units(self) -> dict[str, list[str]]:
return {"kk": ["syncthing.service"]}
+20
View File
@@ -0,0 +1,20 @@
import os
import decman
class Example(decman.Plugin):
NAME = "example"
def available(self) -> bool:
return os.path.exists("/tmp/example_plugin_available")
def process_modules(self, store: decman.Store, modules: list[decman.Module]):
# Toy example for setting modules as changed
for module in modules:
module._changed = True
def apply(
self, store: decman.Store, dry_run: bool = False, params: list[str] | None = None
) -> bool:
return True
+14
View File
@@ -0,0 +1,14 @@
[project]
name = "decman-plugin-example"
version = "0.1.0"
requires-python = ">=3.13"
dependencies = [
"decman",
]
[project.entry-points."decman.plugins"]
example = "decman_plugin_example:Example"
[build-system]
requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"
+13 -259
View File
@@ -1,269 +1,23 @@
# This example covers all decman features and many useful ways of configuring a system.
# Configuration can be as simple or as complex as is needed.
from base import BaseModule
from kde import KDE
import socket
import os
# Note: Do NOT use from imports for global variables
# BAD: from decman import packages/modules/etc
import decman
import decman.config
# This is fine since the thing being imported is a class and not a global variable.
from decman import UserPackage, File, Directory, UserRaisedError
decman.pacman.packages |= {"openssh", "qemu-guest-agent", "sudo", "vim"}
# Configuring what packages are installed is easy.
# Duplicates are OK, so if you have multiple modules that want to ensure a package is installed,
# you can add the same package multiple times.
decman.packages += ["python", "python", "devtools", "git", "networkmanager"]
decman.modules += [BaseModule(), KDE()]
# Decman matches installed packages to those defined in the configuration.
# This means that:
# - all packages not installed on the system but defined in the source are installed
# - all packages installed on the system but not defined in the source are removed
# To make decman not care if a package is installed or not, add it to ignored_packages.
# Ignored packages can be normal packages or aur packages.
decman.ignored_packages += ["rustup", "yay"]
decman.files["/home/arch/.vimrc"] = decman.File(
source_file="./files/vimrc", owner="arch", permissions=0o600
)
# Installing AUR packages is easy.
decman.aur_packages += ["protonvpn"]
# To import GPG keys, set the GNUPGHOME environment variable.
# It can easily be done with python as well.
os.environ["GNUPGHOME"] = "/home/kk/.gnupg/"
# You then must set the user that builds the packages to the owner of the GPG home.
decman.config.makepkg_user = "kk"
# You can also install packages from anywhere, but then you must include some
# information about the package. The git_url is the url to the PKGBUILD,
# This example may not be up to date, but you should keep these up to date with the PKGBUILD.
decman.user_packages.append(
UserPackage(
pkgname="decman-git",
version="0.2.1",
provides=["decman"],
dependencies=[
"python",
"python-requests",
"devtools",
decman.execution_order = [
"files",
"pacman",
"aur",
"flatpak",
"systemd",
"git",
"less",
],
make_dependencies=[
"python-setuptools",
"python-build",
"python-installer",
"python-wheel",
],
git_url="https://github.com/kiviktnm/decman-pkgbuild.git",
))
# Managing only packages with decman is not that interesting.
# Decman also has really powerful ways of managing config files, scripts etc.
# IMPORTANT: Decman will remove files that were created by decman, but are no longer in the decman source.
# Keep your files in version control to avoid losing important files accidentally.
# Define file content inline.
# Default text file encoding is utf-8 but it can be changed.
decman.files["/etc/vconsole.conf"] = File(content="KEYMAP=us",
encoding="utf-8")
# Include file content from another file, set the file owner and permissions.
# The source_file is relative to the directory where the main decman source.py is located.
# By default, the file group is set to the group of the owner, but it can be overridden with the group argument.
decman.files["/home/kk/.bin/user-script.sh"] = File(
source_file="files/user-script.sh", owner="kk", permissions=0o744)
# Non-text files such as images can also be managed.
decman.files["/home/kk/.background.png"] = File(
source_file="files/i-dont-actually-exist.png", bin_file=True, owner="kk")
# If you need to install multiple files at once, use directories.
# All files from the source directory will be copied recursively to the target.
decman.directories["/home/kk/.config/app/"] = Directory(
source_directory="files/app-config", owner="kk")
# Decman has built in support for managing systemd units as well.
# Decman will enable services declared here, and disable services removed from here.
# If you don't want decman to manage a service, don't add it here. It will ignore all units that
# weren't enabled here.
decman.enabled_systemd_units += ["NetworkManager.service"]
# You can manage units for users as well.
# Ensure that previous user unit declarations aren't overwritten and they are initialized.
decman.enabled_systemd_user_units[
"kk"] = decman.enabled_systemd_user_units.get("kk", [])
# Add user unit.
decman.enabled_systemd_user_units["kk"].append("syncthing.service")
# Most powerful feature of decman are modules.
# In this file you see how to include your module, but to really see what modules are capable of
# look at the MyModule class.
from my_module import MyModule
my_own_mod = MyModule()
# You have full access to python, which makes your configuration very dynamic.
# For example: do something if the computers hostname is arch-1
if socket.gethostname() == "arch-1":
# Modules make dynamic configuration easy.
# This executes code defined in MyModule which can affect for example what packages are
# installed as a part of this module.
my_own_mod.enable_my_custom_feature(True)
else:
# If you want to abort running decman from your config because something is wrong, raise a UserRaisedError
raise UserRaisedError("Unknown hostname!")
decman.modules += [my_own_mod]
# Configuring the behavior of decman is also done here.
# These are the default values.
# Note: you probably don't want to change these 2 settings and instead you'll want to to use the --debug CLI option.
# Show debug output
decman.config.debug_output = False
# Suppress output of some commands that you probably don't want to see.
decman.config.suppress_command_output = True
# Make output less verbose. Summaries are still printed.
decman.config.quiet_output = False
# The user which builds aur and user packages.
# decman.config.makepkg_user = "nobody" # This was set in a previous example. Let's not override it.
# The build directory decman uses for creating a chroot etc.
decman.config.build_dir = "/tmp/decman/build"
# Built packages are stored here.
decman.config.pkg_cache_dir = "/var/cache/decman"
# Timeout in seconds for fetching aur package details.
decman.config.aur_rpc_timeout = 30
# Enable installing and upgrading foreign packages.
decman.config.enable_fpm = True
# Number of package files per package kept in the cache
# All built AUR packages and user packages are stored in cache.
decman.config.number_of_packages_stored_in_cache = 3
# Changing the default commands decman uses for things is a bit more complex.
# Create a child class of the decman.config.Commands class and override methods.
# These are the defaults.
class MyCommands(decman.config.Commands):
def list_pkgs(self) -> list[str]:
return ["pacman", "-Qeq", "--color=never"]
def list_foreign_pkgs_versioned(self) -> list[str]:
return ["pacman", "-Qm", "--color=never"]
def install_pkgs(self, pkgs: list[str]) -> list[str]:
return ["pacman", "-S", "--needed"] + pkgs
def install_files(self, pkg_files: list[str]) -> list[str]:
return ["pacman", "-U", "--asdeps"] + pkg_files
def set_as_explicitly_installed(self, pkgs: list[str]) -> list[str]:
return ["pacman", "-D", "--asexplicit"] + pkgs
def install_deps(self, deps: list[str]) -> list[str]:
return ["pacman", "-S", "--needed", "--asdeps"] + deps
def is_installable(self, pkg: str) -> list[str]:
return ["pacman", "-Sddp", pkg]
def upgrade(self) -> list[str]:
return ["pacman", "-Syu"]
def remove(self, pkgs: list[str]) -> list[str]:
return ["pacman", "-Rs"] + pkgs
def enable_units(self, units: list[str]) -> list[str]:
return ["systemctl", "enable"] + units
def disable_units(self, units: list[str]) -> list[str]:
return ["systemctl", "disable"] + units
def enable_user_units(self, units: list[str], user: str) -> list[str]:
return ["systemctl", "--user", "-M", f"{user}@", "enable"] + units
def disable_user_units(self, units: list[str], user: str) -> list[str]:
return ["systemctl", "--user", "-M", f"{user}@", "disable"] + units
def compare_versions(self, installed_version: str,
new_version: str) -> list[str]:
return ["vercmp", installed_version, new_version]
def git_clone(self, repo: str, dest: str) -> list[str]:
return ["git", "clone", repo, dest]
def git_diff(self, from_commit: str) -> list[str]:
return ["git", "diff", from_commit]
def git_get_commit_id(self) -> list[str]:
return ["git", "rev-parse", "HEAD"]
def review_file(self, file: str) -> list[str]:
return ["less", file]
def make_chroot(self, chroot_dir: str, with_pkgs: list[str]) -> list[str]:
return ["mkarchroot", chroot_dir] + with_pkgs
def install_chroot_packages(self, chroot_dir: str, packages: list[str]):
return [
"arch-nspawn", chroot_dir, "pacman", "-S", "--needed",
"--noconfirm"
] + packages
def remove_chroot_packages(self, chroot_dir: str, packages: list[str]):
return ["arch-nspawn", chroot_dir, "pacman", "-Rsu", "--noconfirm"
] + packages
def make_chroot_pkg(self, chroot_wd_dir: str, user: str,
pkgfiles_to_install: list[str]) -> list[str]:
makechrootpkg_cmd = [
"makechrootpkg", "-c", "-r", chroot_wd_dir, "-U", user
]
for pkgfile in pkgfiles_to_install:
makechrootpkg_cmd += ["-I", pkgfile]
return makechrootpkg_cmd
# To apply your overrides, set the commands variable.
decman.config.commands = MyCommands()
# Alternative to the built in AUR support:
# If you don't want to use the built in AUR helper, you can use some pacman wrapper that can run as root, such as pikaur.
# To do this, override commands and disable fpm.
class PikaurWrapperCommands(decman.config.Commands):
def list_pkgs(self) -> list[str]:
return ["pikaur", "-Qeq"]
def install_pkgs(self, pkgs: list[str]) -> list[str]:
return ["pikaur", "-S"] + pkgs
def upgrade(self) -> list[str]:
return ["pikaur", "-Syu"]
def remove(self, pkgs: list[str]) -> list[str]:
return ["pikaur", "-Rs"] + pkgs
# it doesn't matter if all pacman commands aren't overridden since they wont be used when fpm is disabled.
# decman.config.enable_fpm = False
# decman.config.commands = PikaurWrapperCommands()
# Then simply add all AUR packages to decman.packages
# decman.packages += ["pikaur"]
decman.pacman.packages.add("flatpak")
decman.flatpak.packages |= {"org.mozilla.firefox", "org.signal.Signal"}
+20
View File
@@ -0,0 +1,20 @@
[project]
name = "decman-flatpak"
version = "1.1.0"
requires-python = ">=3.13"
dependencies = ["decman==1.2.0"]
[project.entry-points."decman.plugins"]
flatpak = "decman.plugins.flatpak:Flatpak"
[build-system]
requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"
[tool.setuptools.packages.find]
where = ["src"]
namespaces = true
include = ["decman.plugins*"]
[tool.pytest.ini_options]
testpaths = ["tests"]
@@ -0,0 +1,277 @@
import shutil
import decman.core.command as command
import decman.core.error as errors
import decman.core.module as module
import decman.core.output as output
import decman.core.store as _store
import decman.plugins as plugins
def packages(fn):
"""
Annotate that this function returns a set of flatpak package names that should be installed.
Return type of ``fn``: ``set[str]``
"""
fn.__flatpak__packages__ = True
return fn
def user_packages(fn):
"""
Annotate that this function returns a dict of users and flatpak packages that should be
installed.
Return type of ``fn``: ``dict[str, set[str]]``
"""
fn.__flatpak__user__packages__ = True
return fn
class Flatpak(plugins.Plugin):
"""
Plugin that manages flatpak packages added directly to ``packages`` or declared by modules via
``@flatpak.packages``. User packages are managed as well.
"""
NAME = "flatpak"
def __init__(self) -> None:
self.packages: set[str] = set()
self.user_packages: dict[str, set[str]] = {}
self.ignored_packages: set[str] = set()
self.commands = FlatpakCommands()
def available(self) -> bool:
return shutil.which("flatpak") is not None
def process_modules(self, store: _store.Store, modules: list[module.Module]):
# These store keys are used to track changes in modules.
# This way when these change, module can be marked as changed
store.ensure("flatpaks_for_module", {})
store.ensure("user_flatpaks_for_module", {})
for mod in modules:
store["flatpaks_for_module"].setdefault(mod.name, set())
store["user_flatpaks_for_module"].setdefault(mod.name, {})
packages = set().union(
*plugins.run_methods_with_attribute(mod, "__flatpak__packages__")
)
user_packages = {
k: v
for d in plugins.run_methods_with_attribute(mod, "__flatpak__user__packages__")
for k, v in d.items()
}
if store["flatpaks_for_module"][mod.name] != packages:
mod._changed = True
output.print_debug(
f"Module '{mod.name}' set to changed due to modified system flatpaks."
)
if store["user_flatpaks_for_module"][mod.name] != user_packages:
mod._changed = True
output.print_debug(
f"Module '{mod.name}' set to changed due to modified user flatpaks."
)
self.packages |= packages
for user, flatpaks in user_packages.items():
self.user_packages.setdefault(user, set()).update(flatpaks)
store["flatpaks_for_module"][mod.name] = packages
store["user_flatpaks_for_module"][mod.name] = user_packages
def apply(
self, store: _store.Store, dry_run: bool = False, params: list[str] | None = None
) -> bool:
pm = FlatpakInterface(self.commands)
try:
self.apply_packages(pm, None, self.packages, self.ignored_packages, dry_run)
for user, packages in self.user_packages.items():
self.apply_packages(pm, user, packages, self.ignored_packages, dry_run)
except errors.CommandFailedError as error:
output.print_error("Running a flatpak command failed.")
output.print_error(
"Flatpak command exited with an unexpected return code. You may have cancelled a "
"flatpak operation."
)
output.print_error(str(error))
if error.output:
output.print_command_output(error.output)
output.print_traceback()
return False
return True
def apply_packages(
self,
flatpak: "FlatpakInterface",
user: str | None,
packages: set[str],
ignored_packages: set[str],
dry_run: bool,
):
currently_installed = flatpak.get_apps(user)
to_remove = currently_installed - packages - ignored_packages
to_install = packages - currently_installed - ignored_packages
for_user_msg = f" for {user}" if user else ""
if to_remove:
output.print_list(f"Removing flatpak packages{for_user_msg}:", sorted(to_remove))
if not dry_run:
flatpak.remove(to_remove, user)
output.print_summary(f"Upgrading packages{for_user_msg}.")
if not dry_run:
flatpak.upgrade(user)
if to_install:
output.print_list(f"Installing flatpak packages{for_user_msg}:", sorted(to_install))
if not dry_run:
flatpak.install(to_install, user)
class FlatpakCommands:
def list_apps(self, as_user: bool) -> list[str]:
"""
Running this command outputs a newline separated list of installed flatpak application IDs.
If ``as_user`` is ``True``, run the command as the user whose packages should be listed.
NOTE: The first line says 'Application ID' and should be ignored.
"""
return [
"flatpak",
"list",
"--app",
"--user" if as_user else "--system",
"--columns",
"application",
]
def install(self, pkgs: set[str], as_user: bool) -> list[str]:
"""
Running this command installs all listed packages, and their dependencies/runtimes
automatically.
If ``as_user`` is ``True``, run the command as the user for whom packages are installed.
"""
return [
"flatpak",
"install",
"--user" if as_user else "--system",
] + sorted(pkgs)
def upgrade(self, as_user: bool) -> list[str]:
"""
Updates all installed flatpaks including runtimes and dependencies.
If ``as_user`` is ``True``, run the command as the user whose flatpaks are updated.
"""
return [
"flatpak",
"update",
"--user" if as_user else "--system",
]
def remove(self, pkgs: set[str], as_user: bool) -> list[str]:
"""
Running this command will remove the listed packages.
If ``as_user`` is ``True``, run the command as the user for whom packages are removed.
"""
return [
"flatpak",
"remove",
"--user" if as_user else "--system",
] + sorted(pkgs)
def remove_unused(self, as_user: bool) -> list[str]:
"""
This will remove all unused flatpak dependencies and runtimes.
If ``as_user`` is ``True``, run the command as the user for whom packages are removed.
"""
return [
"flatpak",
"remove",
"--unused",
"--user" if as_user else "--system",
]
class FlatpakInterface:
"""
High level interface for running pacman commands.
On failure methods raise a ``CommandFailedError``.
"""
def __init__(self, commands: FlatpakCommands) -> None:
self._commands = commands
def get_apps(self, user: str | None = None) -> set[str]:
"""
Returns a set of installed flatpak apps.
If ``user`` is set, returns flatpak apps for that user.
"""
as_user = user is not None
cmd = self._commands.list_apps(as_user=as_user)
_, packages_text = command.check_run_result(
cmd, command.run(cmd, user=user, mimic_login=as_user)
)
packages = packages_text.strip().split("\n")
# In case no apps are installed, the list contains this
if "" in packages:
packages.remove("")
return set(packages)
def install(self, packages: set[str], user: str | None = None):
"""
Installs the given packages.
If ``user`` is set, installs packages for that user.
"""
if not packages:
return
as_user = user is not None
cmd = self._commands.install(packages, as_user)
command.prg(cmd, user=user, mimic_login=as_user)
def upgrade(self, user: str | None = None):
"""
Upgrades all packages.
If ``user`` is set, upgrades packages for that user.
"""
as_user = user is not None
cmd = self._commands.upgrade(as_user)
command.prg(cmd, user=user, mimic_login=as_user)
def remove(self, packages: set[str], user: str | None = None):
"""
Removes the given packages as well as unused dependencies.
If ``user`` is set, removes packages for that user.
"""
if not packages:
return
as_user = user is not None
cmd = self._commands.remove(packages, as_user)
command.prg(cmd, user=user, mimic_login=as_user)
cmd = self._commands.remove_unused(as_user)
command.prg(cmd, user=user, mimic_login=as_user)
+31
View File
@@ -0,0 +1,31 @@
[project]
name = "decman-pacman"
version = "1.1.0"
requires-python = ">=3.13"
dependencies = [
"decman==1.2.0",
"pyalpm",
"requests",
]
[dependency-groups]
dev = [
"pytest>=8.4.2",
"pytest-mock>=3.15.1",
]
[project.entry-points."decman.plugins"]
pacman = "decman.plugins.pacman:Pacman"
aur = "decman.plugins.aur:AUR"
[build-system]
requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"
[tool.setuptools.packages.find]
where = ["src"]
namespaces = true
include = ["decman.plugins*"]
[tool.pytest.ini_options]
testpaths = ["tests"]
@@ -0,0 +1,261 @@
import os
import shutil
import pyalpm
from decman.plugins.aur.commands import AurCommands, AurPacmanInterface
from decman.plugins.aur.error import (
AurRPCError,
DependencyCycleError,
ForeignPackageManagerError,
PKGBUILDParseError,
)
from decman.plugins.aur.fpm import ForeignPackageManager
from decman.plugins.aur.package import CustomPackage, PackageSearch
import decman.config as config
import decman.core.error as errors
import decman.core.module as module
import decman.core.output as output
import decman.core.store as _store
import decman.plugins as plugins
# Re-exports
__all__ = [
"AUR",
"AurCommands",
"CustomPackage",
"packages",
"custom_packages",
]
def packages(fn):
"""
Annotate that this function returns a set of AUR package names that should be installed.
Return type of ``fn``: ``set[str]``
"""
fn.__aur__packages__ = True
return fn
def custom_packages(fn):
"""
Annotate that this function returns a set of ``CustomPackage``s that should be installed.
Return type of ``fn``: ``set[CustomPackage]``
"""
fn.__custom__packages__ = True
return fn
class AUR(plugins.Plugin):
"""
Plugin that manages additional pacman packages installed outside the pacman repos.
AUR packages are added directly to ``packages`` or declared by modules via ``@aur.packages``.
Custom packages are added directly to ``custom_packages`` or declared by modules via
``@aur.custom_packages``.
"""
NAME = "aur"
def __init__(self) -> None:
self.packages: set[str] = set()
self.custom_packages: set[CustomPackage] = set()
self.ignored_packages: set[str] = set()
self.commands: AurCommands = AurCommands()
self.database_signature_level = pyalpm.SIG_DATABASE_OPTIONAL
self.database_path = "/var/lib/pacman/"
self.aur_rpc_timeout: int = 30
self.print_highlights: bool = True
self.keywords: set[str] = {
"pacsave",
"pacnew",
# These cause too many false positives IMO
# "warning",
# "error",
# "note",
}
self.build_dir: str = "/tmp/decman/build"
self.makepkg_user: str = "nobody"
def available(self) -> bool:
return (
shutil.which("pacman") is not None
and shutil.which("git") is not None
and shutil.which("mkarchroot") is not None
)
def process_modules(self, store: _store.Store, modules: list[module.Module]):
# This is used to track changes in modules.
store.ensure("aur_packages_for_module", {})
store.ensure("custom_packages_for_module", {})
for mod in modules:
store["aur_packages_for_module"].setdefault(mod.name, set())
store["custom_packages_for_module"].setdefault(mod.name, set())
aur_packages = set().union(
*plugins.run_methods_with_attribute(mod, "__aur__packages__")
)
custom_packages = set().union(
*plugins.run_methods_with_attribute(mod, "__custom__packages__")
)
custom_package_strs = set(map(str, custom_packages))
if store["aur_packages_for_module"][mod.name] != aur_packages:
mod._changed = True
output.print_debug(
f"Module '{mod.name}' set to changed due to modified aur packages."
)
if store["custom_packages_for_module"][mod.name] != custom_package_strs:
mod._changed = True
output.print_debug(
f"Module '{mod.name}' set to changed due to modified custom packages."
)
self.packages |= aur_packages
self.custom_packages |= custom_packages
store["aur_packages_for_module"][mod.name] = aur_packages
store["custom_packages_for_module"][mod.name] = custom_package_strs
def apply(
self, store: _store.Store, dry_run: bool = False, params: list[str] | None = None
) -> bool:
params = params or []
upgrade_devel = "aur-upgrade-devel" in params
force = "aur-force" in params
pkg_cache_dir = os.path.join(config.cache_dir, "aur/")
if not dry_run:
try:
os.makedirs(pkg_cache_dir, exist_ok=True)
except OSError as error:
output.print_error(
"Failed to ensure AUR package cache directory exists: "
f"{error.strerror or error}"
)
output.print_traceback()
return False
try:
package_search = PackageSearch(self.aur_rpc_timeout)
for custom_package in self.custom_packages:
package_search.add_custom_pkg(custom_package.parse(self.commands))
pm = AurPacmanInterface(
self.commands,
self.print_highlights,
self.keywords,
self.database_signature_level,
self.database_path,
)
fpm = ForeignPackageManager(
store,
pm,
package_search,
self.commands,
pkg_cache_dir,
self.build_dir,
self.makepkg_user,
)
custom_package_names = {p.pkgname for p in self.custom_packages}
currently_installed_native = pm.get_native_explicit()
currently_installed_foreign = pm.get_foreign_explicit()
orphans = pm.get_foreign_orphans()
to_remove = (
(currently_installed_foreign | orphans)
- self.packages
- custom_package_names
- self.ignored_packages
)
actually_to_remove = set()
to_set_as_dependencies = set()
dependants_to_keep = (
self.packages
| custom_package_names
| currently_installed_native
# don't remove ignored packages' dependencies
| (self.ignored_packages & currently_installed_foreign)
)
for package in to_remove:
dependants = pm.get_dependants(package)
if dependants & dependants_to_keep:
to_set_as_dependencies.add(package)
else:
actually_to_remove.add(package)
if actually_to_remove:
output.print_list("Removing foreign packages:", sorted(actually_to_remove))
if not dry_run:
pm.remove(actually_to_remove)
if to_set_as_dependencies:
output.print_list(
"Setting previously explicitly installed foreign packages as dependencies:",
sorted(to_set_as_dependencies),
)
if not dry_run:
pm.set_as_dependencies(to_set_as_dependencies)
output.print_summary("Upgrading foreign packages.")
if not dry_run:
# don't try to upgrade removed packages
fpm.upgrade(upgrade_devel, force, self.ignored_packages | actually_to_remove)
to_install = (
(self.packages | custom_package_names)
- currently_installed_foreign
- self.ignored_packages
)
output.print_list("Installing foreign packages:", sorted(to_install))
if not dry_run:
fpm.install(list(to_install), force=force)
except AurRPCError as error:
output.print_error("Failed to fetch data from AUR RPC.")
output.print_error(str(error))
output.print_traceback()
return False
except DependencyCycleError as error:
output.print_error("Foreign package dependency cycle detected.")
output.print_error(str(error))
output.print_traceback()
return False
except PKGBUILDParseError as error:
output.print_error("Failed to parse a CustomPackage PKGBUILD.")
output.print_error(str(error))
output.print_traceback()
return False
except ForeignPackageManagerError as error:
output.print_error("Foreign package manager failed.")
output.print_error(str(error))
output.print_traceback()
return False
except pyalpm.error as error:
output.print_error("Failed to query pacman databases with pyalpm.")
output.print_error(str(error))
output.print_traceback()
return False
except errors.CommandFailedError as error:
output.print_error(
"AUR command exited with an unexpected return code. You may have cancelled a "
"pacman operation."
)
output.print_error(str(error))
if error.output:
output.print_command_output(error.output)
output.print_traceback()
return False
return True
@@ -0,0 +1,203 @@
import decman.plugins.pacman as pacman
import decman.config as config
import decman.core.command as command
class AurCommands(pacman.PacmanCommands):
def install_as_dependencies(self, pkgs: set[str]) -> list[str]:
"""
Running this command installs the given packages from pacman repositories.
The packages are installed as dependencies.
"""
return ["pacman", "-S", "--needed", "--asdeps"] + list(pkgs)
def install_files_as_dependencies(self, pkg_files: list[str]) -> list[str]:
"""
Running this command installs the given packages files as dependencies.
"""
return ["pacman", "-U", "--asdeps"] + pkg_files
def compare_versions(self, installed_version: str, new_version: str) -> list[str]:
"""
Running this command outputs -1 when the installed version is older than the new version.
"""
return ["vercmp", installed_version, new_version]
def git_clone(self, repo: str, dest: str) -> list[str]:
"""
Running this command clones a git repository to the the given destination.
"""
return ["git", "clone", repo, dest]
def git_diff(self, from_commit: str) -> list[str]:
"""
Running this command outputs the difference between the given commit and
the current state of the repository.
"""
return ["git", "diff", from_commit]
def git_get_commit_id(self) -> list[str]:
"""
Running this command outputs the current commit id.
"""
return ["git", "rev-parse", "HEAD"]
def git_log_commit_ids(self) -> list[str]:
"""
Running this command outputs commit hashes of the repository.
"""
return ["git", "log", "--format=format:%H"]
def review_file(self, file: str) -> list[str]:
"""
Running this command outputs a file for the user to see.
"""
return ["less", file]
def make_chroot(self, chroot_dir: str, with_pkgs: set[str]) -> list[str]:
"""
Running this command creates a new arch chroot to the chroot directory and installs the
given packages there.
"""
return ["mkarchroot", chroot_dir] + list(with_pkgs)
def install_chroot(self, chroot_dir: str, packages: list[str]):
"""
Running this command installs the given packages to the given chroot.
"""
return [
"arch-nspawn",
chroot_dir,
"pacman",
"-S",
"--needed",
"--noconfirm",
] + packages
def resolve_real_name_chroot(self, chroot_dir: str, pkg: str) -> list[str]:
"""
This command prints a real name of a package.
For example, it prints the package which provides a virtual package.
"""
return [
"arch-nspawn",
chroot_dir,
"pacman",
"-Sddp",
"--print-format=%n",
pkg,
]
def remove_chroot(self, chroot_dir: str, packages: set[str]):
"""
Running this command removes the given packages from the given chroot.
"""
return ["arch-nspawn", chroot_dir, "pacman", "-Rsu", "--noconfirm"] + list(packages)
def make_chroot_pkg(
self, chroot_wd_dir: str, user: str, pkgfiles_to_install: list[str]
) -> list[str]:
"""
Running this command creates a package file using the given chroot.
The package is created as the user and the pkg_files_to_install are installed
in the chroot before the package is created.
"""
makechrootpkg_cmd = ["makechrootpkg", "-c", "-r", chroot_wd_dir, "-U", user]
for pkgfile in pkgfiles_to_install:
makechrootpkg_cmd += ["-I", pkgfile]
return makechrootpkg_cmd
def print_srcinfo(self) -> list[str]:
"""
Running this command prints SRCINFO generated from the package in the current
working directory.
"""
return ["makepkg", "--printsrcinfo"]
class AurPacmanInterface(pacman.PacmanInterface):
"""
High level interface for running pacman commands.
On failure methods raise a ``CommandFailedError``.
"""
def __init__(
self,
commands: AurCommands,
print_highlights: bool,
keywords: set[str],
dbsiglevel: int,
dbpath: str,
) -> None:
super().__init__(commands, print_highlights, keywords, dbsiglevel, dbpath)
self._installable: dict[str, bool] = {}
self._aur_commands = commands
def get_foreign_orphans(self) -> set[str]:
"""
Returns a set of orphaned foreign packages.
"""
return self._get_orphans(pacman.PacmanInterface._is_foreign)
def is_provided_by_installed(self, dependency: str) -> bool:
return pacman.strip_dependency(dependency) in self._local_provides_index
def filter_installed_packages(self, deps: set[str]) -> set[str]:
out = set()
for d in deps:
if not self.is_provided_by_installed(d) and d not in self.get_all_packages():
out.add(d)
return out
def is_installable(self, pkg: str) -> bool:
"""
Returns True if a package can be installed using pacman.
"""
return (
pacman.strip_dependency(pkg) in self._name_index
or pacman.strip_dependency(pkg) in self._provides_index
)
def get_versioned_foreign_packages(self) -> list[tuple[str, str]]:
"""
Returns a list of installed packages and their versions that aren't from pacman databases,
basically AUR packages.
"""
out: list[tuple[str, str]] = []
for pkg in self._handle.get_localdb().pkgcache:
if not self._is_native(pkg.name):
out.append((pkg.name, pkg.version))
return out
def install_dependencies(self, deps: set[str]):
"""
Installs the given dependencies.
"""
if not deps:
return
cmd = self._aur_commands.install_as_dependencies(deps)
pacman_output = command.prg(cmd)
self.print_highlighted_pacman_messages(pacman_output)
def install_files(self, files: list[str], as_explicit: set[str]):
"""
Installs the given files first as dependencies. Then the packages listed in as_explicit are
installed explicitly.
"""
if not files:
return
cmd = self._aur_commands.install_files_as_dependencies(files)
pacman_output = command.prg(cmd)
self.print_highlighted_pacman_messages(pacman_output)
if not as_explicit:
return
cmd = self._commands.set_as_explicit(as_explicit)
command.prg(cmd, pty=config.debug_output)
@@ -0,0 +1,40 @@
class ForeignPackageManagerError(Exception):
"""
Error raised from the ForeignPackageManager
"""
class DependencyCycleError(Exception):
"""
Error raised when a dependency cycle is detected involving foreign packages.
"""
def __init__(self, package1: str, package2: str):
super().__init__(
f"Foreign package dependency cycle detected involving '{package1}' "
f"and '{package2}'. Foreign package dependencies are also required "
"during package building and therefore dependency cycles cannot be handled."
)
class PKGBUILDParseError(Exception):
"""
Error raised when parsing a PKGBUILD fails.
"""
def __init__(self, git_url: str | None, pkgbuild_directory: str | None, message: str) -> None:
# Only one of these should be set
self.pkgbuild_source = git_url or pkgbuild_directory
self.message = message
super().__init__(f"Failed to parse PKGBUILD from '{self.pkgbuild_source}': {message}")
class AurRPCError(Exception):
"""
Error raised when accessing AUR RPC fails.
"""
def __init__(self, message: str, url: str):
self.message = message
self.url = url
super().__init__(f"Failed to complete AUR RPC request to '{url}': {message}")
@@ -0,0 +1,828 @@
import os
import shutil
import time
import typing
from decman.plugins.aur.commands import AurCommands
from decman.plugins.aur.error import ForeignPackageManagerError
from decman.plugins.aur.package import AurPacmanInterface, PackageSearch
from decman.plugins.aur.resolver import DepGraph, ForeignPackage
import decman.config as config
import decman.core.command as command
import decman.core.error as errors
import decman.core.output as output
import decman.core.store as _store
def find_latest_cached_package(store: _store.Store, package: str) -> tuple[str, str] | None:
"""
Returns the latest version and path of a package stored in the built packages cache as a
tuple (version, path).
"""
store.ensure("package_file_cache", {})
entries = store["package_file_cache"].get(package)
if entries is None:
return None
latest_version = None
latest_path = None
latest_timestamp = 0
for version, path, timestamp in entries:
if latest_timestamp < timestamp and os.path.exists(path):
latest_timestamp = timestamp
latest_version = version
latest_path = path
output.print_debug(f"Latest file for {package} is '{latest_path}'.")
if latest_path is None:
return None
assert latest_version is not None, "If latest_path is set, then latest_version is set."
return (latest_version, latest_path)
def add_package_to_cache(store: _store.Store, package: str, version: str, path_to_built_pkg: str):
"""
Adds a built package to the package file cache. Tries to remove excess cached packages.
"""
store.ensure("package_file_cache", {})
new_entry = (version, path_to_built_pkg, int(time.time()))
entries = store["package_file_cache"].get(package, [])
for _, already_cached_path, __ in entries:
if already_cached_path == path_to_built_pkg:
output.print_debug(
f"Trying to cache {package} version {version}, but the version is already cached: "
f"{already_cached_path}"
)
return
entries.append(new_entry)
store["package_file_cache"][package] = entries
clean_package_cache(store, package)
def clean_package_cache(store: _store.Store, package: str):
oldest_path = None
oldest_timestamp = None
index_of_oldest = None
entries = store["package_file_cache"][package]
output.print_debug(f"Package cache has {len(entries)} entries.")
number_of_packages_stored_in_cache = 3
if len(entries) <= number_of_packages_stored_in_cache:
output.print_debug("Old files will not be removed.")
return
for index, entry in enumerate(entries):
_, path, timestamp = entry
if oldest_timestamp is None or oldest_timestamp > timestamp:
oldest_timestamp = timestamp
oldest_path = path
index_of_oldest = index
output.print_debug(f"Oldest cached file for {package} is '{oldest_path}'.")
if oldest_path is None:
return
assert index_of_oldest is not None
entries.pop(index_of_oldest)
if os.path.exists(oldest_path):
output.print_debug(f"Removing '{oldest_path}' from the package cache.")
try:
os.remove(oldest_path)
except OSError as e:
output.print_error(f"Failed to remove file '{oldest_path}' from the package cache.")
output.print_error(e.strerror or str(e))
output.print_error("You'll have to remove the file manually.")
store["package_file_cache"][package] = entries
def is_devel(package: str) -> bool:
"""
Returns True if the given package is a devel package.
"""
devel_suffixes = [
"-git",
"-hg",
"-bzr",
"-svn",
"-cvs",
"-darcs",
]
for suffix in devel_suffixes:
if package.endswith(suffix):
return True
return False
class ResolvedDependencies:
"""
Result of dependency resolution.
"""
def __init__(self) -> None:
self.pacman_deps: set[str] = set()
self.foreign_pkgs: set[str] = set()
self.foreign_dep_pkgs: set[str] = set()
self.foreign_build_dep_pkgs: set[str] = set()
self.build_order: list[str] = []
self.packages: dict[str, ForeignPackage] = {}
# maps dependency names to package names
self.providers: dict[str, list[str]] = {}
self.all_provided: set[str] = set()
self._pkgbases_to_pkgs: dict[str, set[str]] = {}
self._pkgs_to_pkgbases: dict[str, str] = {}
def add_pkgbase_info(self, pkgname: str, pkgbase: str):
"""
Adds information about a which package belongs in which package base.
"""
pkgs = self._pkgbases_to_pkgs.get(pkgbase, set())
pkgs.add(pkgname)
self._pkgbases_to_pkgs[pkgbase] = pkgs
self._pkgs_to_pkgbases[pkgname] = pkgbase
def get_pkgbase(self, pkgname: str) -> str:
"""
Returns the package base of an package.
"""
return self._pkgs_to_pkgbases[pkgname]
def get_pkgs_with_common_pkgbase(self, pkgname: str) -> set[str]:
"""
Returns all packages that have the same package base as the given package.
"""
pkgbase = self._pkgs_to_pkgbases[pkgname]
return self._pkgbases_to_pkgs[pkgbase]
def all_pkgbases(self) -> list[str]:
"""
Returns all pkgbases.
"""
return list(self._pkgbases_to_pkgs)
def get_some_pkgname(self, pkgbase: str) -> str:
"""
Returns some package name that the given pkgbase has.
"""
return list(self._pkgbases_to_pkgs[pkgbase])[0]
class ForeignPackageManager:
"""
Class for dealing with foreign packages.
"""
def __init__(
self,
store: _store.Store,
pacman: AurPacmanInterface,
search: PackageSearch,
commands: AurCommands,
pkg_cache_dir: str,
build_dir: str,
makepkg_user: str,
):
self._store = store
self._pacman = pacman
self._search = search
self._commands = commands
self._pkg_cache_dir = pkg_cache_dir
self._build_dir = build_dir
self._makepkg_user = makepkg_user
def upgrade(
self,
upgrade_devel: bool = False,
force: bool = False,
ignored_pkgs: typing.Optional[set[str]] = None,
):
"""
Upgrades all foreign packages.
"""
if ignored_pkgs is None:
ignored_pkgs = set()
output.print_info("Determining foreign packages to upgrade.")
all_foreign_pkgs = self._pacman.get_versioned_foreign_packages()
all_explicit_foreign_pkgs = set(self._pacman.get_foreign_explicit())
output.print_debug(f"Foreign packages to check for upgrades: {all_foreign_pkgs}")
self._search.try_caching_packages(list(map(lambda p: p[0], all_foreign_pkgs)))
as_explicit = []
as_deps = []
for pkg, ver in all_foreign_pkgs:
if pkg in ignored_pkgs:
continue
info = self._search.get_package_info(pkg)
if info is None:
raise ForeignPackageManagerError(
f"Failed to find '{pkg}' from AUR or user provided packages."
)
if self.should_upgrade_package(pkg, ver, info.version, upgrade_devel):
if pkg in all_explicit_foreign_pkgs:
as_explicit.append(pkg)
else:
as_deps.append(pkg)
output.print_debug(
f"The following foreign packages will be upgraded: {' '.join(as_explicit)}"
)
self.install(as_explicit, as_deps, force)
def install(
self,
foreign_pkgs: list[str],
foreign_dep_pkgs: typing.Optional[list[str]] = None,
force: bool = False,
):
"""
Installs the given foreign packages and their dependencies (both pacman/AUR).
"""
if foreign_dep_pkgs is None:
foreign_dep_pkgs = []
if len(foreign_pkgs) == 0 and len(foreign_dep_pkgs) == 0:
return
resolved_dependencies = self.resolve_dependencies(foreign_pkgs, foreign_dep_pkgs)
output.print_list(
"The following foreign packages will be installed explicitly:",
sorted(resolved_dependencies.foreign_pkgs),
)
output.print_list(
"The following foreign packages will be installed as dependencies:",
sorted(resolved_dependencies.foreign_dep_pkgs),
)
output.print_list(
"The following foreign packages will be built in order to install other packages. "
"They will not be installed:",
sorted(resolved_dependencies.foreign_build_dep_pkgs),
)
if not output.prompt_confirm("Proceed?", default=True):
raise ForeignPackageManagerError("Installing aborted by the user.")
needed_pacman_deps = self._pacman.filter_installed_packages(
resolved_dependencies.pacman_deps - resolved_dependencies.all_provided
)
output.print_summary("Installing foreign package dependencies from pacman.")
self._pacman.install_dependencies(needed_pacman_deps)
try:
with PackageBuilder(
self._search,
self._store,
self._pacman,
resolved_dependencies,
self._commands,
self._pkg_cache_dir,
self._build_dir,
self._makepkg_user,
) as builder:
while resolved_dependencies.build_order:
to_build = resolved_dependencies.build_order.pop(0)
pkgbase = resolved_dependencies.get_pkgbase(to_build)
package_names = resolved_dependencies.get_pkgs_with_common_pkgbase(to_build)
packages = [
resolved_dependencies.packages[pkgname] for pkgname in package_names
]
builder.build_packages(pkgbase, packages, force)
except OSError as e:
raise ForeignPackageManagerError("Failed to build packages.") from e
packages_to_install = resolved_dependencies.foreign_pkgs
packages_to_install |= resolved_dependencies.foreign_dep_pkgs
package_files_to_install = []
for pkg in packages_to_install:
built_pkg = find_latest_cached_package(self._store, pkg)
assert built_pkg is not None
_, path = built_pkg
package_files_to_install.append(path)
if package_files_to_install or force:
output.print_summary("Installing foreign packages.")
self._pacman.install_files(
package_files_to_install,
as_explicit=resolved_dependencies.foreign_pkgs
- resolved_dependencies.foreign_dep_pkgs,
)
else:
output.print_summary("No packages to install.")
def resolve_dependencies(
self,
foreign_pkgs: list[str],
foreign_dep_pkgs: typing.Optional[list[str]] = None,
) -> ResolvedDependencies:
"""
Resolves foreign dependencies of foreign packages.
"""
output.print_info("Resolving foreign package dependencies.")
output.print_debug(f"Packages: {foreign_pkgs}")
if foreign_dep_pkgs is None:
foreign_dep_pkgs = []
result = ResolvedDependencies()
result.foreign_pkgs = set(foreign_pkgs)
result.foreign_dep_pkgs = set(foreign_dep_pkgs)
graph = DepGraph()
for name in foreign_pkgs + foreign_dep_pkgs:
graph.add_requirement(name, None)
seen_packages = set(foreign_pkgs + foreign_dep_pkgs)
to_process = foreign_pkgs + foreign_dep_pkgs
total_processed = 0
self._search.try_caching_packages(to_process)
def process_dep(pkgname: str, depname: str, add_to: set[str]):
dep_info = self._search.find_provider(depname)
if dep_info is None:
raise ForeignPackageManagerError(
f"Failed to find '{depname}' from AUR or user provided packages."
)
add_to.add(dep_info.pkgname)
output.print_debug(f"Adding dependency {dep_info.pkgname} to package {pkgname}.")
graph.add_requirement(dep_info.pkgname, pkgname)
if dep_info.pkgname not in seen_packages:
to_process.append(dep_info.pkgname)
seen_packages.add(dep_info.pkgname)
while to_process:
pkgname = to_process.pop()
info = self._search.get_package_info(pkgname)
if info is None:
raise ForeignPackageManagerError(
f"Failed to find '{pkgname}' from AUR or user provided packages."
)
for provided in info.provides:
result.providers.setdefault(provided, []).append(pkgname)
result.all_provided.add(provided)
result.pacman_deps.update(info.native_dependencies(self._pacman))
result.add_pkgbase_info(pkgname, info.pkgbase)
build_deps = info.foreign_make_dependencies(
self._pacman
) + info.foreign_check_dependencies(self._pacman)
self._search.try_caching_packages(info.foreign_dependencies(self._pacman) + build_deps)
for depname in info.foreign_dependencies(self._pacman):
process_dep(pkgname, depname, result.foreign_dep_pkgs)
for depname in build_deps:
process_dep(pkgname, depname, result.foreign_build_dep_pkgs)
total_processed += 1
output.print_info(f"Progress: {total_processed}/{len(seen_packages)}.")
output.print_info("Determining build order.")
while True:
to_add = graph.get_and_remove_outer_dep_pkgs()
if len(to_add) == 0:
break
for pkg in to_add:
if pkg not in result.packages:
output.print_debug(f"Adding {pkg} to build_order.")
result.build_order.append(pkg.name)
result.packages[pkg.name] = pkg
return result
def should_upgrade_package(
self,
package: str,
installed_version: str,
fetched_version: str,
upgrade_devel=False,
) -> bool:
"""
Returns True if a package should be upgraded.
"""
if upgrade_devel and is_devel(package):
output.print_debug(f"Package {package} is devel package. It should be upgraded.")
return True
try:
cmd = self._commands.compare_versions(installed_version, fetched_version)
vercmp_output = command.prg(cmd, pty=False)
should_upgrade = int(vercmp_output) < 0
output.print_debug(
f"Installed version is: {installed_version}. "
f"Available version is {fetched_version}. Should upgrade: {should_upgrade}."
)
return should_upgrade
except (ValueError, errors.CommandFailedError) as error:
output.print_error(f"{error}")
raise ForeignPackageManagerError("Failed to compare versions using vercmp.") from error
class PackageBuilder:
"""
Used for building packages in a chroot.
"""
always_included_packages = ["base-devel", "git"]
def __init__(
self,
search: PackageSearch,
store: _store.Store,
pacman: AurPacmanInterface,
resolved_deps: ResolvedDependencies,
commands: AurCommands,
pkg_cache_dir: str,
build_dir: str,
makepkg_user: str,
):
self._search = search
self._store = store
self._pacman = pacman
self._resolved_deps = resolved_deps
self._commands = commands
self.pkg_cache_dir = pkg_cache_dir
self.build_dir = build_dir
self.makepkg_user = makepkg_user
self.valid_pkgexts = [
".pkg.tar",
".pkg.tar.gz",
".pkg.tar.bz2",
".pkg.tar.xz",
".pkg.tar.zst",
".pkg.tar.lzo",
".pkg.tar.lrz",
".pkg.tar.lz4",
".pkg.tar.lz",
".pkg.tar.Z",
]
self.chroot_wd_dir = os.path.join(build_dir, "chroot")
self.chroot_dir = os.path.join(self.chroot_wd_dir, "root")
self.pkgbase_dir_map: dict[str, str] = {}
self.original_wd = ""
self._pkgs_in_chroot = set(PackageBuilder.always_included_packages)
self._pkgs_in_chroot.update(resolved_deps.pacman_deps)
def __enter__(self):
self.store_wd()
self.create_build_environment()
return self
def __exit__(self, exc_type, exc_value, traceback):
self.restore_wd()
self.remove_build_environment()
def store_wd(self):
"""
Remembers the current working directory as the original working directory.
"""
self.original_wd = os.getcwd()
def restore_wd(self):
"""
Returns to the original working directory.
"""
os.chdir(self.original_wd)
def create_build_environment(self):
"""
Creates a new chroot and clones all PKGBUILDS.
"""
output.print_info("Creating a build environment..")
if os.path.exists(self.build_dir):
output.print_info("Removing previous build directory.")
self.remove_build_environment()
output.print_info("Getting all PKGBUILDS.")
# Set up PKGBUILDS
for pkgbase in self._resolved_deps.all_pkgbases():
pkgbuild_dir = os.path.join(self.build_dir, pkgbase)
self.pkgbase_dir_map[pkgbase] = pkgbuild_dir
os.makedirs(pkgbuild_dir)
os.chdir(pkgbuild_dir)
pkgbase_info = self._search.get_package_info(
self._resolved_deps.get_some_pkgname(pkgbase)
)
assert pkgbase_info is not None, (
"All dependencies and packages should be resolved "
"during the creation of ResolvedDependencies."
)
output.print_debug(f"Git URL for '{pkgbase}' is '{pkgbase_info.git_url}'")
output.print_debug(
f"PKGBUILD directory for '{pkgbase}' is '{pkgbase_info.pkgbuild_directory}'"
)
self._fetch_and_review_pkgbuild(
pkgbase, pkgbase_info.git_url, pkgbase_info.pkgbuild_directory
)
shutil.chown(pkgbuild_dir, user=self.makepkg_user)
output.print_info("Creating a new chroot.")
os.makedirs(self.chroot_wd_dir)
# Remove GNUPGHOME from mkarchroot environment variables since it may interfere with
# the chroot creation
mkarchroot_env_vars = os.environ.copy()
try:
del mkarchroot_env_vars["GNUPGHOME"]
output.print_debug("Removed GNUPGHOME variable from mkarchroot environment.")
except KeyError:
pass
cmd = self._commands.make_chroot(self.chroot_dir, self._pkgs_in_chroot)
command.prg(
cmd, env_overrides=mkarchroot_env_vars, pass_environment=False, pty=config.debug_output
)
def remove_build_environment(self):
"""
Deletes the build environment.
"""
shutil.rmtree(self.build_dir)
def build_packages(self, package_base: str, packages: list[ForeignPackage], force: bool):
"""
Builds package(s) with the same package base.
Set force to true to force rebuilds of packages that are already cached
"""
package_names = list(map(lambda p: p.name, packages))
# Rebuild is only needed if at least one package is not in the cache.
if self._are_all_pkgs_cached(packages) and not force:
output.print_info(f"Skipped building '{' '.join(package_names)}'. Already up to date.")
return
output.print_info(f"Building '{' '.join(package_names)}'.")
chroot_new_pacman_pkgs, chroot_pkg_files = self._get_chroot_packages(packages)
pkgbuild_dir = self.pkgbase_dir_map[package_base]
os.chdir(pkgbuild_dir)
output.print_debug(f"Chroot dir is: '{self.chroot_dir}', pkgbuild dir is '{pkgbuild_dir}'.")
output.print_info("Installing build dependencies to chroot.")
cmd = self._commands.install_chroot(
self.chroot_dir, chroot_new_pacman_pkgs + PackageBuilder.always_included_packages
)
command.prg(cmd, pty=config.debug_output)
output.print_info("Making package.")
cmd = self._commands.make_chroot_pkg(
self.chroot_wd_dir, self.makepkg_user, chroot_pkg_files
)
command.prg(cmd)
for pkgname in package_names:
file = self._find_pkgfile(pkgname, pkgbuild_dir)
dest = shutil.copy(file, self.pkg_cache_dir)
pkg_info = self._search.get_package_info(pkgname)
# Because all dependencies and packages should be resolved during the creation
# of ResolvedDependencies.
assert pkg_info is not None
version = pkg_info.version
output.print_debug(
f"Adding '{pkgname}', version: '{version}' to cache as file '{dest}'."
)
add_package_to_cache(self._store, pkgname, version, dest)
if len(chroot_new_pacman_pkgs) != 0:
to_remove = set()
for p in chroot_new_pacman_pkgs:
if p not in self._pkgs_in_chroot:
cmd = self._commands.resolve_real_name_chroot(self.chroot_dir, p)
_, cmd_output = command.check_run_result(cmd, command.run(cmd))
real_pkgname = cmd_output.strip()
to_remove.add(real_pkgname)
if to_remove:
output.print_info("Removing build dependencies from chroot.")
cmd = self._commands.remove_chroot(self.chroot_dir, to_remove)
command.prg(cmd, pty=config.debug_output)
else:
output.print_debug("No build dependencies to remove from chroot.")
output.print_info(f"Finished building: '{' '.join(package_names)}'.")
def _are_all_pkgs_cached(self, pkgs: list[ForeignPackage]) -> bool:
for pkg in pkgs:
cache_entry = find_latest_cached_package(self._store, pkg.name)
if cache_entry is None:
return False
cached_version, _ = cache_entry
pkg_info = self._search.get_package_info(pkg.name)
# Because all dependencies and packages should be resolved during the creation
# of ResolvedDependencies. git_url should not be None.
assert pkg_info is not None
fetched_version = pkg_info.version
if cached_version != fetched_version or is_devel(pkg.name):
return False
return True
def _get_chroot_packages(
self, pkgs_to_build: list[ForeignPackage]
) -> tuple[list[str], list[str]]:
"""
Returns a tuple of pacman build dependencies and built foreign pkgs files that are needed
in the chroot before building. pkgs_to_build share the same pkgbase.
"""
chroot_pacman_build_deps = set()
chroot_foreign_pkgs = set()
def add_to_pacman_build_deps(deps: list[str]):
for dep in deps:
if dep not in self._resolved_deps.pacman_deps:
chroot_pacman_build_deps.add(dep)
for pkg in pkgs_to_build:
info = self._search.get_package_info(pkg.name)
# Because all dependencies and packages should be resolved during the creation
# of ResolvedDependencies. git_url should not be None.
assert info is not None
add_to_pacman_build_deps(info.native_make_dependencies(self._pacman))
add_to_pacman_build_deps(info.native_check_dependencies(self._pacman))
foreign_deps = pkg.get_all_recursive_foreign_dep_pkgs()
chroot_foreign_pkgs.update(foreign_deps)
# Add pacman deps of foreign packages
for dep in foreign_deps:
dep_info = self._search.get_package_info(dep)
# Because all dependencies and packages should be resolved during the creation
# of ResolvedDependencies. git_url should not be None.
assert dep_info is not None
add_to_pacman_build_deps(dep_info.native_make_dependencies(self._pacman))
add_to_pacman_build_deps(dep_info.native_check_dependencies(self._pacman))
# Packages with the same pkgbase might depend on each other,
# but they don't need to be installed for the build to succeed.
for pkg in pkgs_to_build:
if pkg.name in chroot_foreign_pkgs:
chroot_foreign_pkgs.remove(pkg.name)
chroot_foreign_pkg_files = []
for foreign_pkg in chroot_foreign_pkgs:
entry = find_latest_cached_package(self._store, foreign_pkg)
assert entry is not None, (
"Build order determines that the dependencies are built "
"before and thus are found in the cache."
)
_, file = entry
chroot_foreign_pkg_files.append(file)
return (list(chroot_pacman_build_deps), chroot_foreign_pkg_files)
def _find_pkgfile(self, pkgname: str, pkgbuild_dir: str) -> str:
# HACK: Because we don't know the pkgarch we can't be sure what is the build result.
# Instead: we just try with pre- and postfixes.
matches = []
info = self._search.get_package_info(pkgname)
assert info is not None
prefix = info.pkg_file_prefix()
for file in os.scandir(pkgbuild_dir):
if file.is_file() and file.name.startswith(prefix):
for ext in self.valid_pkgexts:
if file.name.endswith(ext):
matches.append(file.path)
continue
if len(matches) != 1:
raise ForeignPackageManagerError(
f"Failed to build package '{pkgname}', because the pkg file cannot be determined. "
f"Possible files are: {matches}"
)
return matches[0]
def _fetch_and_review_pkgbuild(
self, pkgbase: str, git_url: str | None, pkgbuild_directory: str | None
):
"""
Fetches a PKGBUILD to the current directory.
PKGBUILD will be cloned using git if ``git_url`` is set.
PKGBUILD will be copied from ``pkgbuild_directory`` if it is set.
The user is prompted to review the PKGBUILD and confirm if the package should be built.
"""
self._store.ensure("pkgbuild_latest_reviewed_commits", {})
if git_url:
cmd = self._commands.git_clone(git_url, ".")
command.prg(cmd, pty=config.debug_output)
if pkgbuild_directory:
try:
shutil.copytree(pkgbuild_directory, ".", dirs_exist_ok=True)
# Chmod to 755 to allow reading files
mode = 0o755
for root, dirs, files in os.walk("."):
for name in dirs + files:
os.chmod(os.path.join(root, name), mode)
os.chmod(".", mode)
except OSError as error:
raise ForeignPackageManagerError(f"Failed to copy {pkgbuild_directory}.") from error
if output.prompt_confirm(f"Review PKGBUILD or show diff for {pkgbase}?", default=True):
latest_reviewed_commit = None
git_commit_ids = []
if git_url:
latest_reviewed_commit = self._store["pkgbuild_latest_reviewed_commits"].get(
pkgbase
)
cmd = self._commands.git_log_commit_ids()
git_output = command.prg(cmd, pty=False)
git_commit_ids = git_output.strip().split("\n")
if latest_reviewed_commit is None or latest_reviewed_commit not in git_commit_ids:
try:
for file in os.scandir("."):
if file.is_file() and not file.name.startswith("."):
cmd = self._commands.review_file(file.path)
command.prg(cmd)
except OSError as error:
raise ForeignPackageManagerError(
f"Failed to review files in directory for {pkgbase}."
) from error
else:
cmd = self._commands.git_diff(latest_reviewed_commit)
command.prg(cmd)
if output.prompt_confirm("Build this package?", default=True):
cmd = self._commands.git_get_commit_id()
rc, git_output = command.run(cmd)
if rc == 0:
commit_id = git_output.strip()
self._store["pkgbuild_latest_reviewed_commits"][pkgbase] = commit_id
else:
output.print_debug(
f"{pkgbase} is not in a git repository. Commit ID cannot be saved."
)
else:
raise ForeignPackageManagerError("Building aborted.")
@@ -0,0 +1,719 @@
import dataclasses
import os
import pathlib
import shutil
import tempfile
import decman.plugins.pacman as pacman_module
import requests # type: ignore
from decman.plugins.aur.commands import AurCommands, AurPacmanInterface
from decman.plugins.aur.error import AurRPCError, PKGBUILDParseError
import decman.config as config
import decman.core.command as command
import decman.core.error as errors
import decman.core.output as output
@dataclasses.dataclass(frozen=True, slots=True)
class PackageInfo:
"""
Immutable description of a package to be built or installed.
This class represents *resolved* package metadata and is intended to be
passed around as pure data.
Exactly one source must be specified:
- ``git_url`` for VCS-based (e.g. AUR) packages
- ``pkgbuild_directory`` for local PKGBUILD-based packages
Invariants:
- ``pkgname`` uniquely identifies the package.
- ``pkgbase`` groups split packages.
- Exactly one of ``git_url`` or ``pkgbuild_directory`` is set.
- All dependency containers are immutable.
This object is safe for hashing, set membership, and reuse across runs.
"""
pkgname: str
pkgbase: str
version: str
git_url: str | None = None
pkgbuild_directory: str | None = None
provides: tuple[str, ...] = dataclasses.field(default_factory=tuple)
dependencies: tuple[str, ...] = dataclasses.field(default_factory=tuple)
make_dependencies: tuple[str, ...] = dataclasses.field(default_factory=tuple)
check_dependencies: tuple[str, ...] = dataclasses.field(default_factory=tuple)
# Caches (excluded from eq/hash)
_native_dependencies: tuple[str, ...] | None = dataclasses.field(
default=None, init=False, repr=False, compare=False
)
_foreign_dependencies: tuple[str, ...] | None = dataclasses.field(
default=None, init=False, repr=False, compare=False
)
_native_make_dependencies: tuple[str, ...] | None = dataclasses.field(
default=None, init=False, repr=False, compare=False
)
_foreign_make_dependencies: tuple[str, ...] | None = dataclasses.field(
default=None, init=False, repr=False, compare=False
)
_native_check_dependencies: tuple[str, ...] | None = dataclasses.field(
default=None, init=False, repr=False, compare=False
)
_foreign_check_dependencies: tuple[str, ...] | None = dataclasses.field(
default=None, init=False, repr=False, compare=False
)
def __post_init__(self) -> None:
if self.git_url is None and self.pkgbuild_directory is None:
raise ValueError("Both git_url and pkgbuild_directory cannot be None.")
if self.git_url is not None and self.pkgbuild_directory is not None:
raise ValueError("Both git_url and pkgbuild_directory cannot be set.")
def pkg_file_prefix(self) -> str:
"""
Returns the beginning of the file created from building this package.
"""
return f"{self.pkgname}-{self.version}"
# --- public API ---------------------------------------------------------
def foreign_dependencies(self, pacman: AurPacmanInterface) -> list[str]:
"""
Returns a list of foreign dependencies of this package.
The dependencies are stripped of their version constraints if there are any.
"""
self._ensure_dependencies_cached(pacman)
assert self._foreign_dependencies is not None
return list(self._foreign_dependencies)
def foreign_make_dependencies(self, pacman: AurPacmanInterface) -> list[str]:
"""
Returns a list of foreign make dependencies of this package.
The dependencies are stripped of their version constraints if there are any.
"""
self._ensure_make_dependencies_cached(pacman)
assert self._foreign_make_dependencies is not None
return list(self._foreign_make_dependencies)
def foreign_check_dependencies(self, pacman: AurPacmanInterface) -> list[str]:
"""
Returns a list of foreign check dependencies of this package.
The dependencies are stripped of their version constraints if there are any.
"""
self._ensure_check_dependencies_cached(pacman)
assert self._foreign_check_dependencies is not None
return list(self._foreign_check_dependencies)
def native_dependencies(self, pacman: AurPacmanInterface) -> list[str]:
"""
Returns a list of native dependencies of this package.
The dependencies are stripped of their version constraints if there are any.
"""
self._ensure_dependencies_cached(pacman)
assert self._native_dependencies is not None
return list(self._native_dependencies)
def native_make_dependencies(self, pacman: AurPacmanInterface) -> list[str]:
"""
Returns a list of native make dependencies of this package.
The dependencies are stripped of their version constraints if there are any.
"""
self._ensure_make_dependencies_cached(pacman)
assert self._native_make_dependencies is not None
return list(self._native_make_dependencies)
def native_check_dependencies(self, pacman: AurPacmanInterface) -> list[str]:
"""
Returns a list of native check dependencies of this package.
The dependencies are stripped of their version constraints if there are any.
"""
self._ensure_check_dependencies_cached(pacman)
assert self._native_check_dependencies is not None
return list(self._native_check_dependencies)
# --- internal helpers ---------------------------------------------------
@staticmethod
def _classify_dependencies(
deps: tuple[str, ...], pacman: AurPacmanInterface
) -> tuple[tuple[str, ...], tuple[str, ...]]:
native: list[str] = []
foreign: list[str] = []
for dependency in deps:
stripped = pacman_module.strip_dependency(dependency)
if pacman.is_installable(dependency):
native.append(stripped)
else:
foreign.append(stripped)
return tuple(native), tuple(foreign)
def _ensure_dependencies_cached(self, pacman: AurPacmanInterface) -> None:
if self._native_dependencies is not None:
return
native, foreign = self._classify_dependencies(self.dependencies, pacman)
object.__setattr__(self, "_native_dependencies", native)
object.__setattr__(self, "_foreign_dependencies", foreign)
def _ensure_make_dependencies_cached(self, pacman: AurPacmanInterface) -> None:
if self._native_make_dependencies is not None:
return
native, foreign = self._classify_dependencies(self.make_dependencies, pacman)
object.__setattr__(self, "_native_make_dependencies", native)
object.__setattr__(self, "_foreign_make_dependencies", foreign)
def _ensure_check_dependencies_cached(self, pacman: AurPacmanInterface) -> None:
if self._native_check_dependencies is not None:
return
native, foreign = self._classify_dependencies(self.check_dependencies, pacman)
object.__setattr__(self, "_native_check_dependencies", native)
object.__setattr__(self, "_foreign_check_dependencies", foreign)
class CustomPackage:
"""
Custom package installed from some other location than the official repos or the AUR.
``pkgname`` is required because the PKGBUILD might be for split packages.
Exactly one of ``git_url`` or ``pkgbuild_directory`` must be provided.
Parameters:
``pkgname``:
Name of the package.
``git_url``:
URL to a git repository containing the PKGBUILD.
``pkgbuild_directory``:
Path to the directory containing the PKGBUILD.
"""
def __init__(
self, pkgname: str, git_url: str | None = None, pkgbuild_directory: str | None = None
) -> None:
if git_url is None and pkgbuild_directory is None:
raise ValueError("Both git_url and pkgbuild_directory cannot be None.")
if git_url is not None and pkgbuild_directory is not None:
raise ValueError("Both git_url and pkgbuild_directory cannot be set.")
self.pkgname = pkgname
self.git_url = git_url
self.pkgbuild_directory = pkgbuild_directory
def parse(self, commands: AurCommands) -> PackageInfo:
"""
Parses this package's PKGBUILD to ``PackageInfo``.
If this fails, raises a ``PKGBUILDParseError``.
"""
if self.pkgbuild_directory is not None:
srcinfo = self._srcinfo_from_pkgbuild_directory(commands)
else:
srcinfo = self._srcinfo_from_git(commands)
return self._parse_srcinfo(srcinfo)
def __eq__(self, other: object) -> bool:
if not isinstance(other, CustomPackage):
return False
return (
self.git_url == other.git_url
and self.pkgbuild_directory == other.pkgbuild_directory
and self.pkgname == other.pkgname
)
def __hash__(self) -> int:
return hash((self.pkgname, self.git_url, self.pkgbuild_directory))
def __str__(self) -> str:
if self.git_url is not None:
return f"CustomPackage(pkgname={self.pkgname}, git_url={self.git_url})"
return (
f"CustomPackage(pkgname={self.pkgname}, pkgbuild_directory={self.pkgbuild_directory})"
)
def _srcinfo_from_pkgbuild_directory(self, commands: AurCommands) -> str:
assert self.pkgbuild_directory is not None, (
"This will not get called if pkgbuild_directory is unset."
)
path = pathlib.Path(self.pkgbuild_directory)
if not path.is_dir():
raise PKGBUILDParseError(
self.git_url,
self.pkgbuild_directory,
f"pkgbuild_directory '{path}' does not exist or is not a directory.",
)
if not (path / "PKGBUILD").exists():
raise PKGBUILDParseError(
self.git_url, self.pkgbuild_directory, f"No PKGBUILD found in '{path}'."
)
# Since makepkg cannot run as root even when just printing the SRCINFO,
# use a tmpdir and the user 'nobody'
try:
with tempfile.TemporaryDirectory(prefix="decman-pkgbuild-") as tmpdir:
shutil.copytree(path, tmpdir, dirs_exist_ok=True)
# Allow the user 'nobody' to use this directory
mode = 0o777
for root, dirs, files in os.walk(tmpdir):
for name in dirs + files:
os.chmod(os.path.join(root, name), mode)
os.chmod(tmpdir, 0o777)
return self._run_makepkg_printsrcinfo(pathlib.Path(tmpdir), commands)
except OSError as error:
raise PKGBUILDParseError(
self.git_url,
self.pkgbuild_directory,
"Failed to create temporary directory for the PKGBUILD.",
) from error
def _srcinfo_from_git(self, commands: AurCommands) -> str:
assert self.git_url is not None, "This will not get called if git_url is unset."
try:
with tempfile.TemporaryDirectory(prefix="decman-pkgbuild-") as tmpdir:
tmp_path = pathlib.Path(tmpdir)
# Allow the user 'nobody' to use this directory
os.chmod(tmpdir, 0o777)
try:
cmd = commands.git_clone(self.git_url, tmpdir)
# Use the user nobody, since that will be used later to generate SRCINFO
command.prg(cmd, user="nobody", pty=config.debug_output)
except errors.CommandFailedError as error:
raise PKGBUILDParseError(
self.git_url,
self.pkgbuild_directory,
"Failed to clone PKGBUILD repository.",
) from error
if not (tmp_path / "PKGBUILD").exists():
raise PKGBUILDParseError(
self.git_url,
self.pkgbuild_directory,
f"Cloned repository '{self.git_url}' does not contain a PKGBUILD.",
)
return self._run_makepkg_printsrcinfo(tmp_path, commands)
except OSError as error:
raise PKGBUILDParseError(
self.git_url,
self.pkgbuild_directory,
"Failed to create temporary directory for the PKGBUILD.",
) from error
def _run_makepkg_printsrcinfo(self, path: pathlib.Path, commands: AurCommands) -> str:
orig_wd = os.getcwd()
try:
os.chdir(path)
cmd = commands.print_srcinfo()
# No need to use the makepkg_user config option here.
# For just printing the SRCINFO, hardcoded 'nobody' works
srcinfo = command.prg(cmd, user="nobody", pty=False)
except errors.CommandFailedError as error:
raise PKGBUILDParseError(
self.git_url, self.pkgbuild_directory, "Failed to generate SRCINFO using makepkg."
) from error
finally:
os.chdir(orig_wd)
return srcinfo
def _parse_srcinfo(self, srcinfo: str) -> PackageInfo:
pkgbase: str | None = None
pkgver: str | None = None
pkgrel: str | None = None
epoch: str | None = None
provides: list[str] = []
# I'm not sure if split packages can have dependencies listed in the base.
# Easy to handle regardless
base_depends: list[str] = []
base_makedepends: list[str] = []
base_checkdepends: list[str] = []
pkg_depends: list[str] = []
pkg_makedepends: list[str] = []
pkg_checkdepends: list[str] = []
current_pkg: str | None = None
found_pkgnames = set()
for raw in srcinfo.splitlines():
line = raw.strip()
if not line or line.startswith("#") or "=" not in line:
continue
key, value = (part.strip() for part in line.split("=", 1))
is_base = current_pkg is None
is_target_pkg = current_pkg == self.pkgname
match key:
case "pkgbase":
pkgbase = value
current_pkg = None
case "pkgname":
current_pkg = value
found_pkgnames.add(value)
case "pkgver":
if pkgver is None or current_pkg == self.pkgname:
pkgver = value
case "pkgrel":
if pkgrel is None or current_pkg == self.pkgname:
pkgrel = value
case "epoch":
if epoch is None or current_pkg == self.pkgname:
epoch = value
case "provides":
if is_target_pkg:
provides.append(value)
case "depends":
if is_base:
base_depends.append(value)
elif is_target_pkg:
pkg_depends.append(value)
case "makedepends":
if is_base:
base_makedepends.append(value)
elif is_target_pkg:
pkg_makedepends.append(value)
case "checkdepends":
if is_base:
base_checkdepends.append(value)
elif is_target_pkg:
pkg_checkdepends.append(value)
case _ if key.startswith("depends") and key.removeprefix("depends_") == config.arch:
if is_base:
base_depends.append(value)
elif is_target_pkg:
pkg_depends.append(value)
case _ if (
key.startswith("makedepends")
and key.removeprefix("makedepends_") == config.arch
):
if is_base:
base_makedepends.append(value)
elif is_target_pkg:
pkg_makedepends.append(value)
case _ if (
key.startswith("checkdepends")
and key.removeprefix("checkdepends_") == config.arch
):
if is_base:
base_checkdepends.append(value)
elif is_target_pkg:
pkg_checkdepends.append(value)
if pkgbase is None or pkgver is None:
raise PKGBUILDParseError(
self.git_url,
self.pkgbuild_directory,
"Missing required fields (pkgbase/pkgver) in SRCINFO.",
)
if self.pkgname not in found_pkgnames:
raise PKGBUILDParseError(
self.git_url,
self.pkgbuild_directory,
f"Package {self.pkgname} not found in SRCINFO. "
f"Packages present: {' '.join(found_pkgnames)}.",
)
version_core = pkgver
if pkgrel is not None:
version_core = f"{version_core}-{pkgrel}"
if epoch is not None:
version = f"{epoch}:{version_core}"
else:
version = version_core
return PackageInfo(
pkgname=self.pkgname,
pkgbase=pkgbase,
version=version,
git_url=self.git_url,
pkgbuild_directory=self.pkgbuild_directory,
provides=tuple(provides),
dependencies=tuple(base_depends + pkg_depends),
make_dependencies=tuple(base_makedepends + pkg_makedepends),
check_dependencies=tuple(base_checkdepends + pkg_checkdepends),
)
class PackageSearch:
"""
Allows searcing for packages / providers from the AUR as well as user defined sources.
Results are cached and custom packages are preferred.
"""
def __init__(self, aur_rpc_timeout: int = 30) -> None:
self._package_cache: dict[str, PackageInfo] = {}
self._selected_providers_cache: dict[str, PackageInfo] = {}
self._all_providers_cache: dict[str, list[str]] = {}
self._custom_packages: list[PackageInfo] = []
self._timeout = aur_rpc_timeout
def add_custom_pkg(self, user_pkg: PackageInfo):
"""
Adds the given package to custom packages.
"""
self._custom_packages.append(user_pkg)
self._cache_pkg(user_pkg)
def _cache_pkg(self, pkg: PackageInfo):
for provided_pkg in pkg.provides:
self._all_providers_cache.setdefault(provided_pkg, []).append(pkg.pkgname)
self._package_cache[pkg.pkgname] = pkg
def try_caching_packages(self, packages: list[str]):
"""
Tries caching the given packages. Virtual packages may not be cached.
This can be used before calling get_package_info or find_provider multiple individual
times, because then those methods don't have to make new AUR RPC requests.
"""
uncached_packages = list(filter(lambda p: p not in self._package_cache, packages))
if len(uncached_packages) == 0:
return
output.print_debug(f"Trying to cache {uncached_packages}.")
max_pkgs_per_request = 200
while uncached_packages:
to_request = map(lambda p: f"arg[]={p}", uncached_packages[:max_pkgs_per_request])
uncached_packages = uncached_packages[max_pkgs_per_request:]
url = f"https://aur.archlinux.org/rpc/v5/info?{'&'.join(to_request)}"
output.print_debug(f"Request URL = {url}")
try:
request = requests.get(url, timeout=self._timeout)
d = request.json()
if d["type"] == "error":
raise AurRPCError(f"AUR RPC returned error: {d['error']}", url)
for result in d["results"]:
pkgname = result["Name"]
if pkgname in self._package_cache:
continue
for user_package in self._custom_packages:
if user_package.pkgname == pkgname:
output.print_debug(f"'{pkgname}' found in custom packages.")
self._cache_pkg(user_package)
break
else: # if not in user_packages then:
info = PackageInfo(
pkgname=result["Name"],
pkgbase=result["PackageBase"],
version=result["Version"],
dependencies=result.get("Depends", []),
make_dependencies=result.get("MakeDepends", []),
check_dependencies=result.get("CheckDepends", []),
provides=result.get("Provides", []),
git_url=f"https://aur.archlinux.org/{result['PackageBase']}.git",
)
self._cache_pkg(info)
output.print_debug("Request completed.")
except (requests.RequestException, KeyError) as e:
raise AurRPCError(
f"Failed to fetch package information for {uncached_packages} from AUR RPC.",
url,
) from e
def get_package_info(self, package: str) -> PackageInfo | None:
"""
Returns information about a package.
If the package is not custom, fetches information from the AUR.
Returns None if no such AUR package exists.
"""
output.print_debug(f"Getting info for package '{package}'.")
if package in self._package_cache:
output.print_debug(f"'{package}' found in cache.")
return self._package_cache[package]
# This code is probably not needed since all user packages should be cached
for user_package in self._custom_packages:
if user_package.pkgname == package:
output.print_debug(f"'{package}' found in custom packages.")
self._cache_pkg(user_package)
return user_package
url = f"https://aur.archlinux.org/rpc/v5/info/{package}"
output.print_debug(f"Requesting info for '{package}' from AUR. URL = {url}")
try:
request = requests.get(url, timeout=self._timeout)
d = request.json()
if d["type"] == "error":
raise AurRPCError(f"AUR RPC returned error: {d['error']}", url)
if d["resultcount"] == 0:
output.print_debug(f"'{package}' not found.")
return None
output.print_debug(f"'{package}' found from AUR.")
result = d["results"][0]
info = PackageInfo(
pkgname=result["Name"],
pkgbase=result["PackageBase"],
version=result["Version"],
dependencies=result.get("Depends", []),
make_dependencies=result.get("MakeDepends", []),
check_dependencies=result.get("CheckDepends", []),
provides=result.get("Provides", []),
git_url=f"https://aur.archlinux.org/{result['PackageBase']}.git",
)
self._cache_pkg(info)
return info
except (requests.RequestException, KeyError) as e:
raise AurRPCError(
f"Failed to fetch package information for {package} from AUR RPC.",
url,
) from e
def find_provider(self, stripped_dependency: str) -> PackageInfo | None:
"""
Finds a provider for a dependency. The dependency should not contain version constraints.
May prompt the user to select if multiple are available.
"""
output.print_debug(f"Finding provider for '{stripped_dependency}'.")
if stripped_dependency in self._selected_providers_cache:
output.print_debug(f"'{stripped_dependency}' found in cache.")
return self._selected_providers_cache[stripped_dependency]
output.print_debug("Are there exact name matches?")
exact_name_match = self.get_package_info(stripped_dependency)
if exact_name_match is not None:
output.print_debug("Exact name match found.")
self._selected_providers_cache[stripped_dependency] = exact_name_match
return exact_name_match
output.print_debug("No exact name matches found. Finding providers.")
known_pkg_results = self._all_providers_cache.get(stripped_dependency, [])
for user_package in self._custom_packages:
if (
stripped_dependency in user_package.provides
and stripped_dependency not in known_pkg_results
):
known_pkg_results.append(user_package.pkgname)
if len(known_pkg_results) == 1:
pkg = self.get_package_info(known_pkg_results[0])
assert pkg is not None
output.print_debug(
f"Single provider for '{stripped_dependency}' found in known packages: '{pkg}'."
)
self._selected_providers_cache[stripped_dependency] = pkg
return pkg
if len(known_pkg_results) > 1:
return self._choose_provider(stripped_dependency, known_pkg_results, "user packages")
url = f"https://aur.archlinux.org/rpc/v5/search/{stripped_dependency}?by=provides"
output.print_debug(
f"Requesting providers for '{stripped_dependency}' from AUR. URL = {url}"
)
try:
request = requests.get(url, timeout=self._timeout)
d = request.json()
if d["type"] == "error":
raise AurRPCError(f"AUR RPC returned error: {d['error']}", url)
if d["resultcount"] == 0:
output.print_debug(f"'{stripped_dependency}' not found.")
return None
results = list(map(lambda r: r["Name"], d["results"]))
if len(results) == 1:
pkgname = results[0]
output.print_debug(
f"Single provider for '{stripped_dependency}' found from AUR: '{pkgname}'"
)
info = self.get_package_info(pkgname)
return info
return self._choose_provider(stripped_dependency, results, "AUR")
except (requests.RequestException, KeyError) as e:
raise AurRPCError(
f"Failed to search for {stripped_dependency} from AUR RPC.",
url,
) from e
def _choose_provider(
self, dep: str, possible_providers: list[str], where: str
) -> PackageInfo | None:
min_selection = 1
max_selection = len(possible_providers)
output.print_summary(f"Found {len(possible_providers)} providers for {dep} from {where}.")
providers = "Providers: "
for index, name in enumerate(possible_providers):
providers += f"{index + 1}:{name} "
output.print_summary(providers)
selection = output.prompt_number(
f"Select a provider [{min_selection}-{max_selection}] (default: {min_selection}): ",
min_selection,
max_selection,
default=min_selection,
)
info = self.get_package_info(possible_providers[selection - 1])
if info is not None:
self._selected_providers_cache[dep] = info
return info
@@ -0,0 +1,119 @@
import typing
from decman.plugins.aur.error import DependencyCycleError
class ForeignPackage:
"""
Class used to keep track of foreign recursive dependency packages of an foreign package.
"""
def __init__(self, name: str):
self.name = name
self._all_recursive_foreign_deps: set[str] = set()
def __eq__(self, value: object, /) -> bool:
if isinstance(value, self.__class__):
return (
self.name == value.name
and self._all_recursive_foreign_deps == value._all_recursive_foreign_deps
)
return False
def __hash__(self) -> int:
return self.name.__hash__()
def __repr__(self) -> str:
return f"{self.name}: {{{' '.join(self._all_recursive_foreign_deps)}}}"
def __str__(self) -> str:
return f"{self.name}"
def add_foreign_dependency_packages(self, package_names: typing.Iterable[str]):
"""
Adds dependencies to the package.
"""
self._all_recursive_foreign_deps.update(package_names)
def get_all_recursive_foreign_dep_pkgs(self) -> set[str]:
"""
Returns all dependencies and sub-dependencies of the package.
"""
return set(self._all_recursive_foreign_deps)
class DepNode:
"""
A Node of the DepGraph
"""
def __init__(self, package: ForeignPackage) -> None:
self.parents: dict[str, DepNode] = {}
self.children: dict[str, DepNode] = {}
self.pkg = package
def is_pkgname_in_parents_recursive(self, pkgname: str) -> bool:
"""
Returns True if the given package name is in the parents of this DepNode.
"""
for name, parent in self.parents.items():
if name == pkgname or parent.is_pkgname_in_parents_recursive(pkgname):
return True
return False
class DepGraph:
"""
Represents a graph between foreign packages
"""
def __init__(self) -> None:
self.package_nodes: dict[str, DepNode] = {}
self._childless_node_names: set[str] = set()
def add_requirement(self, child_pkgname: str, parent_pkgname: typing.Optional[str]):
"""
Adds a connection between two packages, creating the child package if it doesn't exist.
The parent is the package that requires the child package.
"""
child_node = self.package_nodes.get(child_pkgname, DepNode(ForeignPackage(child_pkgname)))
self.package_nodes[child_pkgname] = child_node
if len(child_node.children) == 0:
self._childless_node_names.add(child_pkgname)
if parent_pkgname is None:
return
parent_node = self.package_nodes[parent_pkgname]
if parent_node.is_pkgname_in_parents_recursive(child_pkgname):
raise DependencyCycleError(child_pkgname, parent_pkgname)
parent_node.children[child_pkgname] = child_node
child_node.parents[parent_pkgname] = parent_node
if parent_pkgname in self._childless_node_names:
self._childless_node_names.remove(parent_pkgname)
def get_and_remove_outer_dep_pkgs(self) -> list[ForeignPackage]:
"""
Returns all childless nodes of the dependency package graph and removes them.
"""
new_childless_node_names = set()
result = []
for childless_node_name in self._childless_node_names:
childless_node = self.package_nodes[childless_node_name]
for parent in childless_node.parents.values():
new_deps = childless_node.pkg.get_all_recursive_foreign_dep_pkgs()
new_deps.add(childless_node.pkg.name)
parent.pkg.add_foreign_dependency_packages(new_deps)
del parent.children[childless_node_name]
if len(parent.children) == 0:
new_childless_node_names.add(parent.pkg.name)
result.append(childless_node.pkg)
self._childless_node_names = new_childless_node_names
return result
@@ -0,0 +1,412 @@
import re
import shutil
from typing import Callable
import pyalpm
import decman.config as config
import decman.core.command as command
import decman.core.error as errors
import decman.core.module as module
import decman.core.output as output
import decman.core.store as _store
import decman.plugins as plugins
def packages(fn):
"""
Annotate that this function returns a set of pacman package names that should be installed.
Return type of ``fn``: ``set[str]``
"""
fn.__pacman__packages__ = True
return fn
def strip_dependency(dep: str) -> str:
"""
Removes version spefications from a dependency name.
"""
rx = re.compile("(=.*|>.*|<.*)")
return rx.sub("", dep)
class Pacman(plugins.Plugin):
"""
Plugin that manages pacman packages added directly to ``packages`` or declared by modules via
``@packages``.
"""
NAME = "pacman"
def __init__(self) -> None:
self.packages: set[str] = set()
self.ignored_packages: set[str] = set()
self.commands = PacmanCommands()
self.print_highlights = True
self.keywords = {
"pacsave",
"pacnew",
# These cause too many false positives IMO
# "warning",
# "error",
# "note",
}
self.database_signature_level = pyalpm.SIG_DATABASE_OPTIONAL
self.database_path = "/var/lib/pacman/"
def available(self) -> bool:
return shutil.which("pacman") is not None
def process_modules(self, store: _store.Store, modules: list[module.Module]):
# This is used to track changes in modules.
store.ensure("packages_for_module", {})
for mod in modules:
store["packages_for_module"].setdefault(mod.name, set())
packages = set().union(*plugins.run_methods_with_attribute(mod, "__pacman__packages__"))
if store["packages_for_module"][mod.name] != packages:
mod._changed = True
output.print_debug(
f"Module '{mod.name}' set to changed due to modified pacman packages."
)
self.packages |= packages
store["packages_for_module"][mod.name] = packages
def apply(
self, store: _store.Store, dry_run: bool = False, params: list[str] | None = None
) -> bool:
try:
pm = PacmanInterface(
self.commands,
self.print_highlights,
self.keywords,
self.database_signature_level,
self.database_path,
)
currently_installed_native = pm.get_native_explicit()
currently_installed_foreign = pm.get_foreign_explicit()
orphans = pm.get_native_orphans()
to_remove = (
(currently_installed_native | orphans) - self.packages - self.ignored_packages
)
actually_to_remove = set()
to_set_as_dependencies = set()
dependants_to_keep = self.packages | currently_installed_foreign
for package in to_remove:
dependants = pm.get_dependants(package)
if dependants & dependants_to_keep:
to_set_as_dependencies.add(package)
else:
actually_to_remove.add(package)
if actually_to_remove:
output.print_list("Removing pacman packages:", sorted(actually_to_remove))
if not dry_run:
pm.remove(actually_to_remove)
if to_set_as_dependencies:
output.print_list(
"Setting previously explicitly installed packages as dependencies:",
sorted(to_set_as_dependencies),
)
if not dry_run:
pm.set_as_dependencies(to_set_as_dependencies)
output.print_summary("Upgrading packages.")
if not dry_run:
pm.upgrade()
to_install = self.packages - currently_installed_native - self.ignored_packages
output.print_list("Installing pacman packages:", sorted(to_install))
if not dry_run:
pm.install(to_install)
except pyalpm.error as error:
output.print_error("Failed to query pacman databases with pyalpm.")
output.print_error(str(error))
output.print_traceback()
return False
except errors.CommandFailedError as error:
output.print_error(
"Pacman command exited with an unexpected return code. You may have cancelled a "
"pacman operation."
)
output.print_error(str(error))
if error.output:
output.print_command_output(error.output)
output.print_traceback()
return False
return True
class PacmanCommands:
def list_pacman_repos(self) -> list[str]:
"""
Running this command prints a newline seperated list of pacman repositories.
"""
return ["pacman-conf", "--repo-list"]
def install(self, pkgs: set[str]) -> list[str]:
"""
Running this command installs the given packages from pacman repositories.
"""
return ["pacman", "-S", "--needed"] + list(pkgs)
def upgrade(self) -> list[str]:
"""
Running this command upgrades all pacman packages from pacman repositories.
"""
return ["pacman", "-Syu"]
def set_as_dependencies(self, pkgs: set[str]) -> list[str]:
"""
Running this command sets the given packages as dependencies.
"""
return ["pacman", "-D", "--asdeps"] + list(pkgs)
def set_as_explicit(self, pkgs: set[str]) -> list[str]:
"""
Running this command sets the given as explicitly installed.
"""
return ["pacman", "-D", "--asexplicit"] + list(pkgs)
def remove(self, pkgs: set[str]) -> list[str]:
"""
Running this command removes the given packages and their dependencies
(that aren't required by other packages).
"""
return ["pacman", "-Rs"] + list(pkgs)
class PacmanInterface:
"""
High level interface for running pacman commands.
On failure methods raise a ``CommandFailedError`` or ``pyalpm.error``.
"""
def __init__(
self,
commands: PacmanCommands,
print_highlights: bool,
keywords: set[str],
dbsiglevel: int,
dbpath: str,
) -> None:
self._commands = commands
self._print_highlights = print_highlights
self._keywords = keywords
self._dbsiglevel = dbsiglevel
self._dbpath = dbpath
self._handle = self._create_pyalpm_handle()
self._name_index = self._create_name_index()
self._local_provides_index = self._create_local_provides_index()
self._provides_index = self._create_provides_index()
self._requiredby_index = self._create_requiredby_index()
def _create_pyalpm_handle(self):
root = "/"
h = pyalpm.Handle(root, self._dbpath)
cmd = self._commands.list_pacman_repos()
repos = command.prg(cmd, pty=False).strip().split("\n")
# Empty string means no DBs
if "" in repos and len(repos) == 1:
return
for repo in repos:
h.register_syncdb(repo, self._dbsiglevel)
return h
def _create_name_index(self) -> dict[str, pyalpm.Package]:
return {pkg.name: pkg for db in self._handle.get_syncdbs() for pkg in db.pkgcache}
def _create_local_provides_index(self) -> dict[str, set[str]]:
out: dict[str, set[str]] = {}
for pkg in self._handle.get_localdb().pkgcache:
for p in pkg.provides:
out.setdefault(strip_dependency(p), set()).add(pkg.name)
out.setdefault(p, set()).add(pkg.name)
return out
def _create_provides_index(self) -> dict[str, set[str]]:
out: dict[str, set[str]] = {}
for db in self._handle.get_syncdbs():
for pkg in db.pkgcache:
for p in pkg.provides:
out.setdefault(strip_dependency(p), set()).add(pkg.name)
out.setdefault(p, set()).add(pkg.name)
return out
def _create_requiredby_index(self) -> dict[str, set[str]]:
return {p.name: set(p.compute_requiredby()) for p in self._handle.get_localdb().pkgcache}
def _is_native(self, package: str) -> bool:
return package in self._name_index
def _is_foreign(self, package: str) -> bool:
return not self._is_native(package)
def get_all_packages(self) -> set[str]:
"""
Returns a set of all installed packages.
"""
return {pkg for pkg in self._handle.get_localdb().pkgcache}
def get_native_explicit(self) -> set[str]:
"""
Returns a set of explicitly installed native packages.
"""
out: set[str] = set()
for pkg in self._handle.get_localdb().pkgcache:
if pkg.reason == pyalpm.PKG_REASON_EXPLICIT and self._is_native(pkg.name):
out.add(pkg.name)
return out
return packages
def _get_orphans(self, filter_fn: Callable[["PacmanInterface", str], bool]) -> set[str]:
orphans: set[str] = {
p.name
for p in self._handle.get_localdb().pkgcache
if p.reason == pyalpm.PKG_REASON_DEPEND and filter_fn(self, p.name)
}
# Prune orphans until there are only packages that are requiredby other orphans
changed = True
while changed:
changed = False
for name in tuple(orphans):
if self._requiredby_index.get(name, set()) - orphans:
orphans.remove(name)
changed = True
return orphans
def get_native_orphans(self) -> set[str]:
"""
Returns a set of orphaned native packages.
"""
return self._get_orphans(PacmanInterface._is_native)
def get_foreign_explicit(self) -> set[str]:
"""
Returns a set of explicitly installed foreign packages.
"""
out: set[str] = set()
for pkg in self._handle.get_localdb().pkgcache:
if pkg.reason == pyalpm.PKG_REASON_EXPLICIT and not self._is_native(pkg.name):
out.add(pkg.name)
return out
def get_dependants(self, package: str) -> set[str]:
"""
Returns a set of installed packages that depend on the given package.
Includes the package itself.
"""
local = self._handle.get_localdb()
seen: set[str] = set()
stack = [package]
while stack:
name = stack.pop()
if name in seen:
continue
seen.add(name)
pkg = local.get_pkg(name)
if pkg is None:
continue
for dep in pkg.compute_requiredby():
if dep not in seen:
stack.append(dep)
return seen
def set_as_dependencies(self, packages: set[str]):
"""
Marks the given packages as dependency packages.
"""
if not packages:
return
cmd = self._commands.set_as_dependencies(packages)
command.prg(cmd, pty=config.debug_output)
def install(self, packages: set[str]):
"""
Installs the given packages. If the packages are already installed, marks them as
explicitly installed.
"""
if not packages:
return
cmd = self._commands.install(packages)
pacman_output = command.prg(cmd)
self.print_highlighted_pacman_messages(pacman_output)
cmd = self._commands.set_as_explicit(packages)
command.prg(cmd, pty=config.debug_output)
def upgrade(self):
"""
Upgrades all packages.
"""
cmd = self._commands.upgrade()
pacman_output = command.prg(cmd)
self.print_highlighted_pacman_messages(pacman_output)
def remove(self, packages: set[str]):
"""
Removes the given packages.
"""
if not packages:
return
cmd = self._commands.remove(packages)
pacman_output = command.prg(cmd)
self.print_highlighted_pacman_messages(pacman_output)
def print_highlighted_pacman_messages(self, pacman_output: str):
"""
Prints lines that contain pacman output keywords.
"""
if not self._print_highlights:
return
lines = pacman_output.split("\n")
highlight_lines = []
for index, line in enumerate(lines):
for keyword in self._keywords:
if keyword.lower() in line.lower():
highlight_lines.append(f"lines: {index}-{index + 2}")
if index >= 1:
highlight_lines.append(lines[index - 1])
highlight_lines.append(line)
if index + 1 < len(lines):
highlight_lines.append(lines[index + 1])
highlight_lines.append("")
# Break, as to not print the same line again if it contains multiple keywords
break
if highlight_lines:
output.print_summary("Pacman output highlights:")
for line in highlight_lines:
if line.startswith("lines:"):
output.print_summary(line)
else:
output.print_continuation(line)
@@ -0,0 +1,314 @@
from typing import Any
import pytest
from decman.plugins import aur as aur_plugin
class FakeStore(dict):
def ensure(self, key: str, default: Any) -> None:
if key not in self:
self[key] = default
class FakeModule:
def __init__(self, name: str, aur_pkgs: set[str], custom_pkgs: set[Any]) -> None:
self.name = name
self._changed = False
self._aur_pkgs = aur_pkgs
self._custom_pkgs = custom_pkgs
class FakeCustomPackage:
def __init__(self, pkgname: str) -> None:
self.pkgname = pkgname
def __hash__(self) -> int: # needed because instances go into sets
return hash(self.pkgname)
def __eq__(self, other: object) -> bool:
return isinstance(other, FakeCustomPackage) and self.pkgname == other.pkgname
def parse(self, commands: Any) -> str:
# Whatever ForeignPackageManager expects; we just need something to feed into add_custom_pkg
return f"parsed-{self.pkgname}"
def test_process_modules_collects_aur_and_custom_packages_and_marks_changed(
monkeypatch: pytest.MonkeyPatch,
) -> None:
aur = aur_plugin.AUR()
store = FakeStore()
cp1 = FakeCustomPackage("custom1")
cp2 = FakeCustomPackage("custom2")
mod1 = FakeModule("mod1", {"aur1", "aur2"}, {cp1})
mod2 = FakeModule("mod2", {"aur3"}, {cp2})
def fake_run_methods_with_attribute(mod: FakeModule, attr: str):
if attr == "__aur__packages__":
return [mod._aur_pkgs]
if attr == "__custom__packages__":
return [mod._custom_pkgs]
return []
monkeypatch.setattr(
aur_plugin.plugins, "run_methods_with_attribute", fake_run_methods_with_attribute
)
aur.process_modules(store, {mod1, mod2})
# union of all aur/custom packages collected
assert aur.packages == {"aur1", "aur2", "aur3"}
assert aur.custom_packages == {cp1, cp2}
# stored per-module
assert store["aur_packages_for_module"]["mod1"] == {"aur1", "aur2"}
assert store["aur_packages_for_module"]["mod2"] == {"aur3"}
assert store["custom_packages_for_module"]["mod1"] == {str(cp1)}
assert store["custom_packages_for_module"]["mod2"] == {str(cp2)}
# first run: modules marked changed
assert mod1._changed is True
assert mod2._changed is True
def test_apply_respects_ignored_packages_and_protects_their_dependencies(
monkeypatch: pytest.MonkeyPatch,
) -> None:
aur = aur_plugin.AUR()
store = FakeStore()
# Desired AUR/custom state
aur.packages = {"desired-aur"}
cp = FakeCustomPackage("custom-aur")
aur.custom_packages = {cp}
# Ignored foreign package (installed) and an ignored but *uninstalled* package
aur.ignored_packages = {"ignored-aur", "ignored-not-installed"}
# Fake PackageSearch
class FakePackageSearch:
def __init__(self, timeout: int) -> None:
self.timeout = timeout
self.added: list[Any] = []
def add_custom_pkg(self, parsed: Any) -> None:
self.added.append(parsed)
monkeypatch.setattr(aur_plugin, "PackageSearch", FakePackageSearch)
monkeypatch.setattr(aur_plugin.os, "makedirs", lambda *x, **kw: None)
# Fake pacman interface for foreign/native info
class FakePM:
def __init__(self, commands, print_highlights, keywords, dbsiglevel, dbpath) -> None:
self.commands = commands
self.print_highlights = print_highlights
self.keywords = keywords
self.remove_called_with: set[str] | None = None
self.set_as_deps_called_with: set[str] | None = None
def get_native_explicit(self) -> set[str]:
# no natives needed for this scenario
return set()
def get_foreign_explicit(self) -> set[str]:
# All explicitly installed foreign packages:
# - ignored-aur (ignored, must stay and protect deps)
# - dep-of-ignored (candidate; has ignored dependant)
# - orphan-foreign (candidate; no dependants)
return {"ignored-aur", "dep-of-ignored", "orphan-foreign"}
def get_foreign_orphans(self) -> set[str]:
# orphan-foreign also considered orphan
return {"orphan-foreign"}
def get_dependants(self, pkg: str) -> set[str]:
if pkg == "dep-of-ignored":
# ignored-aur depends on dep-of-ignored -> must demote, not remove
return {"ignored-aur"}
if pkg == "orphan-foreign":
return set()
return set()
def remove(self, pkgs: set[str]) -> None:
self.remove_called_with = pkgs
def set_as_dependencies(self, pkgs: set[str]) -> None:
self.set_as_deps_called_with = pkgs
fake_pm = FakePM(None, None, None, None, None)
def fake_pm_ctor(
commands,
print_highlights,
keywords,
dbsiglevel,
dbpath,
) -> FakePM:
fake_pm.commands = commands
fake_pm.print_highlights = print_highlights
fake_pm.keywords = keywords
return fake_pm
monkeypatch.setattr(aur_plugin, "AurPacmanInterface", fake_pm_ctor)
# Fake ForeignPackageManager
class FakeFPM:
def __init__(
self,
store_arg,
pm_arg,
package_search_arg,
commands_arg,
cache_dir,
build_dir,
makepkg_user,
) -> None:
self.store = store_arg
self.pm = pm_arg
self.package_search = package_search_arg
self.commands = commands_arg
self.cache_dir = cache_dir
self.build_dir = build_dir
self.makepkg_user = makepkg_user
self.upgrade_args: tuple[bool, bool, set[str]] | None = None
self.install_called_with: list[str] | None = None
def upgrade(self, upgrade_devel: bool, force: bool, ignored: set[str]) -> None:
self.upgrade_args = (upgrade_devel, force, ignored)
def install(self, pkgs: list[str], force: bool = False) -> None:
# store as set to ignore ordering
self.install_called_with = pkgs
fake_fpm = FakeFPM(None, None, None, None, None, None, None)
def fake_fpm_ctor(
store_arg,
pm_arg,
package_search_arg,
commands_arg,
cache_dir,
build_dir,
makepkg_user,
):
fake_fpm.store = store_arg
fake_fpm.pm = pm_arg
fake_fpm.package_search = package_search_arg
fake_fpm.commands = commands_arg
fake_fpm.cache_dir = cache_dir
fake_fpm.build_dir = build_dir
fake_fpm.makepkg_user = makepkg_user
return fake_fpm
monkeypatch.setattr(aur_plugin, "ForeignPackageManager", fake_fpm_ctor)
printed_lists: list[tuple[str, list[str]]] = []
printed_summaries: list[str] = []
def fake_print_list(title: str, items: list[str]) -> None:
printed_lists.append((title, items))
def fake_print_summary(msg: str) -> None:
printed_summaries.append(msg)
monkeypatch.setattr(aur_plugin.output, "print_list", fake_print_list)
monkeypatch.setattr(aur_plugin.output, "print_summary", fake_print_summary)
# Use params to test flag propagation into upgrade/install
ok = aur.apply(store, dry_run=False, params=["aur-upgrade-devel", "aur-force"])
assert ok is True
# Removal / demotion logic:
#
# custom_package_names = {"custom-aur"}
# currently_installed_foreign = {"ignored-aur", "dep-of-ignored", "orphan-foreign"}
# orphans = {"orphan-foreign"}
#
# to_remove candidates:
# (foreign | orphans) - desired - custom - ignored
# = {"ignored-aur", "dep-of-ignored", "orphan-foreign"} {"orphan-foreign"}
# - {"desired-aur"} - {"custom-aur"} - {"ignored-aur"}
# = {"dep-of-ignored", "orphan-foreign"}
#
# dependants_to_keep includes ignored installed foreign -> dep-of-ignored is demoted, orphan-foreign removed.
assert fake_pm.remove_called_with == {"orphan-foreign"}
assert fake_pm.set_as_deps_called_with == {"dep-of-ignored"}
# Ensure ignored packages were not removed
assert "ignored-aur" not in (fake_pm.remove_called_with or set())
# Upgrade called with flags and ignored set
assert fake_fpm.upgrade_args == (
True,
True,
aur.ignored_packages | (fake_pm.remove_called_with or set()),
)
# to_install = (packages | custom_names) - installed_foreign - ignored
# = {"desired-aur", "custom-aur"} - {"ignored-aur", "dep-of-ignored", "orphan-foreign"}
# - {"ignored-aur", "ignored-not-installed"}
# = {"desired-aur", "custom-aur"}
assert set(fake_fpm.install_called_with or []) == {"desired-aur", "custom-aur"}
# ignored packages must not be installed
assert "ignored-aur" not in (fake_fpm.install_called_with or [])
assert "ignored-not-installed" not in (fake_fpm.install_called_with or [])
# Also check the printed lists mirror this
titles = [t for t, _ in printed_lists]
assert "Removing foreign packages:" in titles
assert "Setting previously explicitly installed foreign packages as dependencies:" in titles
assert "Installing foreign packages:" in titles
remove_list = next(items for t, items in printed_lists if "Removing foreign packages:" in t)
demote_list = next(
items
for t, items in printed_lists
if "Setting previously explicitly installed foreign packages as dependencies:" in t
)
install_list = next(items for t, items in printed_lists if "Installing foreign packages:" in t)
assert remove_list == ["orphan-foreign"]
assert demote_list == ["dep-of-ignored"]
# Order of install_list is deterministic because sorted() is used
assert install_list == ["custom-aur", "desired-aur"]
assert any("Upgrading foreign packages." in s for s in printed_summaries)
def test_apply_returns_false_on_aur_rpc_error(monkeypatch: pytest.MonkeyPatch) -> None:
aur = aur_plugin.AUR()
store = FakeStore()
# Force PackageSearch to fail immediately
class FailingPackageSearch:
def __init__(self, timeout: int) -> None:
raise aur_plugin.AurRPCError("RPC down", "url")
monkeypatch.setattr(aur_plugin, "PackageSearch", FailingPackageSearch)
monkeypatch.setattr(aur_plugin.os, "makedirs", lambda *x, **kw: None)
errors_logged: list[str] = []
continuations: list[str] = []
traceback_called: list[bool] = []
def fake_print_error(msg: str) -> None:
errors_logged.append(msg)
def fake_print_traceback() -> None:
traceback_called.append(True)
monkeypatch.setattr(aur_plugin.output, "print_error", fake_print_error)
monkeypatch.setattr(aur_plugin.output, "print_traceback", fake_print_traceback)
ok = aur.apply(store, dry_run=False)
assert ok is False
assert any("AUR RPC" in msg or "fetch data from AUR RPC" in msg for msg in errors_logged)
assert any("RPC down" in msg for msg in errors_logged)
assert traceback_called
@@ -0,0 +1,743 @@
import pathlib
import pytest
from decman.plugins.aur import package as pkg_mod
from decman.plugins.aur.error import AurRPCError, PKGBUILDParseError
from decman.plugins.aur.package import (
CustomPackage,
PackageInfo,
PackageSearch,
)
@pytest.fixture(autouse=True)
def silence_output(monkeypatch):
# Avoid real I/O / prompts in tests by default
monkeypatch.setattr(pkg_mod.output, "print_debug", lambda *a, **k: None)
monkeypatch.setattr(pkg_mod.output, "print_summary", lambda *a, **k: None)
monkeypatch.setattr(
pkg_mod.output,
"prompt_number",
lambda *a, **k: 1, # safe default
)
# --- PackageInfo -----------------------------------------------------------
def test_packageinfo_requires_exactly_one_source():
with pytest.raises(ValueError, match="cannot be None"):
PackageInfo(pkgname="a", pkgbase="a", version="1.0")
with pytest.raises(ValueError, match="cannot be set"):
PackageInfo(
pkgname="a",
pkgbase="a",
version="1.0",
git_url="git://example",
pkgbuild_directory="/tmp",
)
class DummyPacman:
def __init__(self, installable: set[str]):
self._installable = installable
self.calls: list[str] = []
def is_installable(self, name: str) -> bool:
self.calls.append(name)
return name in self._installable
def _make_pkg_for_deps() -> PackageInfo:
return PackageInfo(
pkgname="pkg",
pkgbase="pkg",
version="1.0",
git_url="git://example",
dependencies=("native>=1", "foreign=2"),
make_dependencies=("make-native", "make-foreign>=3"),
check_dependencies=("check-foreign<4", "check-native"),
)
def test_packageinfo_foreign_and_native_dependencies_are_split_and_stripped():
pacman = DummyPacman(
{
"native>=1",
"make-native",
"check-native",
}
)
pkg = _make_pkg_for_deps()
assert pkg.native_dependencies(pacman) == ["native"]
assert pkg.foreign_dependencies(pacman) == ["foreign"]
assert pkg.native_make_dependencies(pacman) == ["make-native"]
assert pkg.foreign_make_dependencies(pacman) == ["make-foreign"]
assert pkg.native_check_dependencies(pacman) == ["check-native"]
assert pkg.foreign_check_dependencies(pacman) == ["check-foreign"]
# --- CustomPackage ---------------------------------------------------------
def test_custompackage_requires_exactly_one_source():
with pytest.raises(ValueError, match="cannot be None"):
CustomPackage("pkg", git_url=None, pkgbuild_directory=None)
with pytest.raises(ValueError, match="cannot be set"):
CustomPackage("pkg", git_url="git://example", pkgbuild_directory="/tmp")
class DummyCommands:
"""Minimal stub; only here so type checks pass where needed."""
pass
@pytest.mark.parametrize(
"srcinfo, expected_version",
[
(
"""
pkgbase = foo
pkgver = 1.2.3
pkgrel = 4
pkgname = foo
""",
"1.2.3-4",
),
(
"""
pkgbase = foo
pkgver = 1.2.3
pkgrel = 4
epoch = 2
pkgname = foo
""",
"2:1.2.3-4",
),
(
"""
pkgbase = foo
pkgver = 1.2.3
pkgname = foo
""",
"1.2.3",
),
],
)
def test_parse_srcinfo_version_handling(srcinfo: str, expected_version: str) -> None:
pkg = CustomPackage(pkgname="foo", git_url=None, pkgbuild_directory="/dummy")
info = pkg._parse_srcinfo(srcinfo)
assert info.pkgname == "foo"
assert info.pkgbase == "foo"
assert info.version == expected_version
def test_parse_srcinfo_single_package_dependencies() -> None:
srcinfo = """
pkgbase = foo
pkgver = 1.2.3
pkgrel = 1
depends = bar>=1.0
makedepends = baz
checkdepends = qux
pkgname = foo
"""
pkg = CustomPackage(pkgname="foo", git_url=None, pkgbuild_directory="/dummy")
info = pkg._parse_srcinfo(srcinfo)
assert info.dependencies == ("bar>=1.0",)
assert info.make_dependencies == ("baz",)
assert info.check_dependencies == ("qux",)
def test_parse_srcinfo_split_package_uses_only_target_pkg_dependencies(monkeypatch) -> None:
# Ensure arch-specific keys match
monkeypatch.setattr(pkg_mod.config, "arch", "x86_64", raising=False)
srcinfo = """
pkgbase = clion
pkgver = 2025.3
pkgrel = 1
makedepends = rsync
depends = base-dep
depends_x86_64 = base-arch-dep
pkgname = clion
depends = libdbusmenu-glib
depends_x86_64 = clion-arch-dep
checkdepends = clion-check
pkgname = clion-jre
depends = jre-dep
makedepends = jre-make
pkgname = clion-cmake
depends = cmake-dep
"""
pkg = CustomPackage(pkgname="clion", git_url=None, pkgbuild_directory="/dummy")
info = pkg._parse_srcinfo(srcinfo)
# version
assert info.pkgbase == "clion"
assert info.version == "2025.3-1"
# base deps + target pkg deps (including arch-specific)
assert info.dependencies == (
"base-dep",
"base-arch-dep",
"libdbusmenu-glib",
"clion-arch-dep",
)
# only base and target pkg makedepends
assert info.make_dependencies == ("rsync",)
# base + target pkg checkdepends
assert info.check_dependencies == ("clion-check",)
def test_parse_srcinfo_arch_specific_ignored_for_other_arch(monkeypatch) -> None:
# Different arch → *_x86_64 keys should be ignored
monkeypatch.setattr(pkg_mod.config, "arch", "aarch64", raising=False)
srcinfo = """
pkgbase = foo
pkgver = 1.0
pkgrel = 1
depends_x86_64 = base-arch-dep
pkgname = foo
depends = common-dep
depends_x86_64 = pkg-arch-dep
"""
pkg = CustomPackage(pkgname="foo", git_url=None, pkgbuild_directory="/dummy")
info = pkg._parse_srcinfo(srcinfo)
# Only common deps, no *_x86_64 because arch != x86_64
assert info.dependencies == ("common-dep",)
def test_parse_srcinfo_missing_required_fields_raises() -> None:
# Missing pkgbase
srcinfo_no_pkgbase = """
pkgver = 1.0
pkgrel = 1
pkgname = foo
"""
pkg = CustomPackage(pkgname="foo", git_url=None, pkgbuild_directory="/dummy")
with pytest.raises(PKGBUILDParseError) as excinfo:
pkg._parse_srcinfo(srcinfo_no_pkgbase)
assert "pkgbase/pkgver" in str(excinfo.value)
# Missing pkgver
srcinfo_no_pkgver = """
pkgbase = foo
pkgname = foo
"""
with pytest.raises(PKGBUILDParseError) as excinfo2:
pkg._parse_srcinfo(srcinfo_no_pkgver)
assert "pkgbase/pkgver" in str(excinfo2.value)
def test_parse_srcinfo_missing_target_pkg_raises() -> None:
srcinfo = """
pkgbase = foo
pkgver = 1.0
pkgrel = 1
pkgname = other
"""
pkg = CustomPackage(pkgname="foo", git_url=None, pkgbuild_directory="/dummy")
with pytest.raises(PKGBUILDParseError) as excinfo:
pkg._parse_srcinfo(srcinfo)
msg = str(excinfo.value)
assert "Package foo not found in SRCINFO" in msg
assert "other" in msg # listed in present packages
def test_srcinfo_from_pkgbuild_directory_missing_dir_raises(tmp_path: pathlib.Path) -> None:
missing = tmp_path / "does-not-exist"
pkg = CustomPackage(pkgname="foo", git_url=None, pkgbuild_directory=str(missing))
with pytest.raises(PKGBUILDParseError) as excinfo:
pkg._srcinfo_from_pkgbuild_directory(DummyCommands())
msg = str(excinfo.value)
assert "does not exist or is not a directory" in msg
def test_srcinfo_from_pkgbuild_directory_missing_pkgbuild_raises(tmp_path: pathlib.Path) -> None:
path = tmp_path / "pkgdir"
path.mkdir()
pkg = CustomPackage(pkgname="foo", git_url=None, pkgbuild_directory=str(path))
with pytest.raises(PKGBUILDParseError) as excinfo:
pkg._srcinfo_from_pkgbuild_directory(DummyCommands())
msg = str(excinfo.value)
assert "No PKGBUILD found" in msg
def test_custom_package_equality_and_hash() -> None:
a1 = CustomPackage(
pkgname="foo", git_url="https://example.com/repo.git", pkgbuild_directory=None
)
a2 = CustomPackage(
pkgname="foo", git_url="https://example.com/repo.git", pkgbuild_directory=None
)
b = CustomPackage(pkgname="foo", git_url=None, pkgbuild_directory="/some/path")
assert a1 == a2
assert hash(a1) == hash(a2)
assert a1 != b
assert hash(a1) != hash(b)
def test_custom_package_str_git_and_directory() -> None:
git_pkg = CustomPackage(
pkgname="foo",
git_url="https://example.com/repo.git",
pkgbuild_directory=None,
)
dir_pkg = CustomPackage(
pkgname="foo",
git_url=None,
pkgbuild_directory="/some/path",
)
assert "pkgname=foo" in str(git_pkg)
assert "git_url=https://example.com/repo.git" in str(git_pkg)
assert "pkgname=foo" in str(dir_pkg)
assert "pkgbuild_directory=/some/path" in str(dir_pkg)
# --- PackageSearch: caching ------------------------------------------------
def _make_pkg(name: str = "pkg") -> PackageInfo:
return PackageInfo(
pkgname=name,
pkgbase=name,
version="1.0",
git_url=f"git://example/{name}",
provides=("virt-" + name,),
dependencies=("dep",),
make_dependencies=(),
check_dependencies=(),
)
def test_add_custom_pkg_caches_package():
search = PackageSearch()
pkg = _make_pkg("foo")
search.add_custom_pkg(pkg)
assert pkg in search._custom_packages
assert search._package_cache["foo"] is pkg
assert search._all_providers_cache["virt-foo"] == ["foo"]
def test_try_caching_packages_skips_already_cached(monkeypatch):
search = PackageSearch()
pkg = _make_pkg("foo")
search._cache_pkg(pkg)
calls = []
def fake_get(*args, **kwargs):
calls.append((args, kwargs))
raise AssertionError("requests.get should not be called")
monkeypatch.setattr(pkg_mod.requests, "get", fake_get)
search.try_caching_packages(["foo"])
assert calls == []
def test_try_caching_packages_caches_from_aur(monkeypatch):
search = PackageSearch()
def fake_get(url, timeout):
class Resp:
def json(self):
return {
"type": "success",
"results": [
{
"Name": "bar",
"PackageBase": "bar-base",
"Version": "2.0",
"Depends": ["dep1"],
"MakeDepends": ["make1"],
"CheckDepends": ["check1"],
"Provides": ["virt-bar"],
}
],
}
return Resp()
monkeypatch.setattr(pkg_mod.requests, "get", fake_get)
search.try_caching_packages(["bar"])
assert "bar" in search._package_cache
info = search._package_cache["bar"]
assert isinstance(info, PackageInfo)
assert search._all_providers_cache["virt-bar"] == ["bar"]
def test_try_caching_packages_aur_returns_error(monkeypatch):
search = PackageSearch()
def fake_get(url, timeout):
class Resp:
def json(self):
return {"type": "error", "error": "boom"}
return Resp()
monkeypatch.setattr(pkg_mod.requests, "get", fake_get)
with pytest.raises(AurRPCError):
search.try_caching_packages(["bar"])
def test_try_caching_packages_request_exception_raises_aur_error(monkeypatch):
search = PackageSearch()
class DummyError(pkg_mod.requests.RequestException):
pass
def fake_get(url, timeout):
raise DummyError("boom")
monkeypatch.setattr(pkg_mod.requests, "get", fake_get)
with pytest.raises(AurRPCError):
search.try_caching_packages(["bar"])
# --- PackageSearch: get_package_info --------------------------------------
def test_get_package_info_returns_from_cache():
search = PackageSearch()
pkg = _make_pkg("foo")
search._cache_pkg(pkg)
result = search.get_package_info("foo")
assert result is pkg
def test_get_package_info_returns_custom_package_if_not_cached():
search = PackageSearch()
pkg = _make_pkg("foo")
search._custom_packages.append(pkg)
result = search.get_package_info("foo")
assert result is pkg
assert search._package_cache["foo"] is pkg
def test_get_package_info_aur_not_found_returns_none(monkeypatch):
search = PackageSearch()
def fake_get(url, timeout):
class Resp:
def json(self):
return {"type": "success", "resultcount": 0, "results": []}
return Resp()
monkeypatch.setattr(pkg_mod.requests, "get", fake_get)
result = search.get_package_info("foo")
assert result is None
assert "foo" not in search._package_cache
def test_get_package_info_aur_success_caches_and_returns(monkeypatch):
search = PackageSearch()
def fake_get(url, timeout):
class Resp:
def json(self):
return {
"type": "success",
"resultcount": 1,
"results": [
{
"Name": "foo",
"PackageBase": "foo-base",
"Version": "1.2",
"Depends": ["dep1"],
"MakeDepends": ["make1"],
"CheckDepends": ["check1"],
"Provides": ["virt-foo"],
}
],
}
return Resp()
monkeypatch.setattr(pkg_mod.requests, "get", fake_get)
result = search.get_package_info("foo")
assert isinstance(result, PackageInfo)
assert result.pkgname == "foo"
assert search._package_cache["foo"] is result
def test_get_package_info_aur_returns_error(monkeypatch):
search = PackageSearch()
def fake_get(url, timeout):
class Resp:
def json(self):
return {"type": "error", "error": "boom"}
return Resp()
monkeypatch.setattr(pkg_mod.requests, "get", fake_get)
with pytest.raises(AurRPCError):
search.get_package_info("foo")
def test_get_package_info_request_exception_raises_aur_error(monkeypatch):
search = PackageSearch()
class DummyError(pkg_mod.requests.RequestException):
pass
def fake_get(url, timeout):
raise DummyError("boom")
monkeypatch.setattr(pkg_mod.requests, "get", fake_get)
with pytest.raises(AurRPCError):
search.get_package_info("foo")
# --- PackageSearch: find_provider -----------------------------------------
def test_find_provider_uses_selected_providers_cache():
search = PackageSearch()
pkg = _make_pkg("foo")
search._selected_providers_cache["dep"] = pkg
result = search.find_provider("dep")
assert result is pkg
def test_find_provider_exact_name_match(monkeypatch):
search = PackageSearch()
pkg = _make_pkg("dep")
def fake_get_package_info(name: str):
assert name == "dep"
return pkg
monkeypatch.setattr(search, "get_package_info", fake_get_package_info)
result = search.find_provider("dep")
assert result is pkg
assert search._selected_providers_cache["dep"] is pkg
def test_find_provider_single_known_provider(monkeypatch):
search = PackageSearch()
pkg = _make_pkg("provider")
search._all_providers_cache["dep"] = ["provider"]
def fake_get_package_info(name: str):
if name == "dep":
return None
assert name == "provider"
return pkg
monkeypatch.setattr(search, "get_package_info", fake_get_package_info)
result = search.find_provider("dep")
assert result is pkg
assert search._selected_providers_cache["dep"] is pkg
def test_find_provider_aur_search_not_found(monkeypatch):
search = PackageSearch()
def fake_get_package_info(name: str):
# Exact name match should fail
return None
monkeypatch.setattr(search, "get_package_info", fake_get_package_info)
def fake_get(url, timeout):
class Resp:
def json(self):
return {"type": "success", "resultcount": 0, "results": []}
return Resp()
monkeypatch.setattr(pkg_mod.requests, "get", fake_get)
result = search.find_provider("dep")
assert result is None
def test_find_provider_aur_search_single_result(monkeypatch):
search = PackageSearch()
pkg = _make_pkg("provider")
def fake_get_package_info(name: str):
# first call for stripped_dependency -> None
if name == "dep":
return None
assert name == "provider"
return pkg
monkeypatch.setattr(search, "get_package_info", fake_get_package_info)
def fake_get(url, timeout):
class Resp:
def json(self):
return {
"type": "success",
"resultcount": 1,
"results": [{"Name": "provider"}],
}
return Resp()
monkeypatch.setattr(pkg_mod.requests, "get", fake_get)
result = search.find_provider("dep")
assert result is pkg
def test_find_provider_aur_search_multiple_results_calls_choose_provider(monkeypatch):
search = PackageSearch()
def fake_get_package_info(name: str):
# no exact match
return None
monkeypatch.setattr(search, "get_package_info", fake_get_package_info)
def fake_get(url, timeout):
class Resp:
def json(self):
return {
"type": "success",
"resultcount": 2,
"results": [{"Name": "a"}, {"Name": "b"}],
}
return Resp()
monkeypatch.setattr(pkg_mod.requests, "get", fake_get)
sentinel = object()
def fake_choose(dep, providers, where):
assert dep == "dep"
assert providers == ["a", "b"]
assert where == "AUR"
return sentinel
monkeypatch.setattr(search, "_choose_provider", fake_choose)
result = search.find_provider("dep")
assert result is sentinel
def test_find_provider_aur_search_error(monkeypatch):
search = PackageSearch()
def fake_get_package_info(name: str):
return None
monkeypatch.setattr(search, "get_package_info", fake_get_package_info)
def fake_get(url, timeout):
class Resp:
def json(self):
return {"type": "error", "error": "boom"}
return Resp()
monkeypatch.setattr(pkg_mod.requests, "get", fake_get)
with pytest.raises(AurRPCError):
search.find_provider("dep")
def test_find_provider_aur_search_request_exception_raises_aur_error(monkeypatch):
search = PackageSearch()
def fake_get_package_info(name: str):
return None
monkeypatch.setattr(search, "get_package_info", fake_get_package_info)
class DummyError(pkg_mod.requests.RequestException):
pass
def fake_get(url, timeout):
raise DummyError("boom")
monkeypatch.setattr(pkg_mod.requests, "get", fake_get)
with pytest.raises(AurRPCError):
search.find_provider("dep")
# --- PackageSearch: _choose_provider --------------------------------------
def test_choose_provider_prompts_and_caches(monkeypatch):
search = PackageSearch()
providers = ["a", "b", "c"]
selected_pkg = _make_pkg("b")
# override prompt to select "2" (provider "b")
monkeypatch.setattr(
pkg_mod.output,
"prompt_number",
lambda *a, **k: 2,
)
def fake_get_package_info(name: str):
assert name == "b"
return selected_pkg
monkeypatch.setattr(search, "get_package_info", fake_get_package_info)
result = search._choose_provider("dep", providers, "AUR")
assert result is selected_pkg
assert search._selected_providers_cache["dep"] is selected_pkg
@@ -0,0 +1,87 @@
import pytest
from decman.plugins.aur.error import DependencyCycleError
from decman.plugins.aur.resolver import DepGraph, ForeignPackage
def test_add_dependency():
graph = DepGraph()
graph.add_requirement("A", None)
graph.add_requirement("B1", "A")
graph.add_requirement("B2", "A")
graph.add_requirement("C", "B1")
assert "B1" in graph.package_nodes["A"].children
assert "B2" in graph.package_nodes["A"].children
assert "C" in graph.package_nodes["B1"].children
def test_cyclic_dependency_raises():
graph = DepGraph()
graph.add_requirement("A", None)
graph.add_requirement("B", "A")
graph.add_requirement("C", "B")
with pytest.raises(DependencyCycleError):
graph.add_requirement("A", "C")
def _build_graph_for_outer_deps() -> DepGraph:
graph = DepGraph()
# Roots
graph.add_requirement("A", None)
graph.add_requirement("V", None)
# Level B
graph.add_requirement("B1", "A")
graph.add_requirement("B2", "A")
graph.add_requirement("B3", "A")
# Extra dependency B1 -> B2
graph.add_requirement("B1", "B2")
# Level C
graph.add_requirement("C1", "B1")
graph.add_requirement("C2", "B1")
# Level D + cycle-ish edges
graph.add_requirement("D", "C1")
graph.add_requirement("C2", "D")
# Foreign packages and their foreign deps
defs = {
"V": [],
"A": ["B1", "B2", "B3", "C1", "C2", "D"],
"B1": ["C1", "C2", "D"],
"B2": ["B1", "C1", "C2", "D"],
"B3": [],
"C1": ["D", "C2"],
"C2": [],
"D": ["C2"],
}
for name, deps in defs.items():
pkg = ForeignPackage(name)
pkg.add_foreign_dependency_packages(deps)
return graph
def _assert_outer_dep_names(graph: DepGraph, expected: set[str]) -> None:
result = graph.get_and_remove_outer_dep_pkgs()
names = {pkg.name for pkg in result}
assert names == expected
def test_get_and_remove_outer_deps_sequence():
graph = _build_graph_for_outer_deps()
_assert_outer_dep_names(graph, {"C2", "B3", "V"})
_assert_outer_dep_names(graph, {"D"})
_assert_outer_dep_names(graph, {"C1"})
_assert_outer_dep_names(graph, {"B1"})
_assert_outer_dep_names(graph, {"B2"})
_assert_outer_dep_names(graph, {"A"})
_assert_outer_dep_names(graph, set())
@@ -0,0 +1,313 @@
from typing import Any
import pytest
from decman.plugins import pacman as pacman_plugin
@pytest.mark.parametrize(
"dep,expected",
[
("foo", "foo"),
("foo=1.0", "foo"),
("bar>=2", "bar"),
("baz<3", "baz"),
("multi=1.0-2", "multi"),
],
)
def test_strip_dependency(dep, expected):
assert pacman_plugin.strip_dependency(dep) == expected
class FakeStore(dict):
def ensure(self, key: str, default: Any) -> None:
if key not in self:
self[key] = default
class FakeModule:
def __init__(self, name: str, packages: set[str]) -> None:
self.name = name
self._changed = False
self._packages = packages
def test_process_modules_collects_packages_and_marks_changed(
monkeypatch: pytest.MonkeyPatch,
) -> None:
pacman = pacman_plugin.Pacman()
store = FakeStore()
mod1 = FakeModule("mod1", {"pkg1", "pkg2"})
mod2 = FakeModule("mod2", {"pkg3"})
def fake_run_methods_with_attribute(mod: FakeModule, attr: str) -> set[str]:
assert attr == "__pacman__packages__"
return [mod._packages]
monkeypatch.setattr(
pacman_plugin.plugins,
"run_methods_with_attribute",
fake_run_methods_with_attribute,
)
pacman.process_modules(store, {mod1, mod2})
# packages collected
assert pacman.packages == {"pkg1", "pkg2", "pkg3"}
# stored mapping per module
assert store["packages_for_module"]["mod1"] == {"pkg1", "pkg2"}
assert store["packages_for_module"]["mod2"] == {"pkg3"}
# modules marked changed (first run)
assert mod1._changed is True
assert mod2._changed is True
def test_apply_dry_run_computes_sets_and_does_not_call_pacman(
monkeypatch: pytest.MonkeyPatch,
) -> None:
pacman = pacman_plugin.Pacman()
store = FakeStore()
# Desired state
pacman.packages = {"keep-explicit", "new-pkg"}
# Fake PacmanInterface returned by plugin module
class FakePM:
def __init__(
self, commands, print_highlights, keywords, database_signature_level, database_path
) -> None: # noqa: D401
self.commands = commands
self.print_highlights = print_highlights
self.keywords = keywords
self.remove_called_with: set[str] | None = None
self.set_as_deps_called_with: set[str] | None = None
self.upgrade_called = False
self.install_called_with: set[str] | None = None
def get_native_explicit(self) -> set[str]:
# keep-explicit (in desired), old-explicit (to demote/remove)
return {"keep-explicit", "old-explicit"}
def get_foreign_explicit(self) -> set[str]:
# foreign-package protects its deps
return {"foreign-pkg"}
def get_native_orphans(self) -> set[str]:
# orphan-explicit is also candidate
return {"orphan-explicit"}
def get_dependants(self, pkg: str) -> set[str]:
# old-explicit has a foreign dependant -> demote to dep
# orphan-explicit has no dependants -> remove
if pkg == "old-explicit":
return {"foreign-pkg"}
if pkg == "orphan-explicit":
return set()
return set()
def remove(self, pkgs: set[str]) -> None:
self.remove_called_with = pkgs
def set_as_dependencies(self, pkgs: set[str]) -> None:
self.set_as_deps_called_with = pkgs
def upgrade(self) -> None:
self.upgrade_called = True
def install(self, pkgs: set[str]) -> None:
self.install_called_with = pkgs
fake_pm = FakePM(None, None, None, None, None)
def fake_pm_ctor(
commands, print_highlights, keywords, database_signature_level, database_path
) -> FakePM:
# constructor used in Pacman.apply
fake_pm.commands = commands
fake_pm.print_highlights = print_highlights
fake_pm.keywords = keywords
return fake_pm
monkeypatch.setattr(pacman_plugin, "PacmanInterface", fake_pm_ctor)
printed_lists: list[tuple[str, list[str]]] = []
printed_summaries: list[str] = []
def fake_print_list(title: str, items: list[str]) -> None:
printed_lists.append((title, items))
def fake_print_summary(msg: str) -> None:
printed_summaries.append(msg)
monkeypatch.setattr(pacman_plugin.output, "print_list", fake_print_list)
monkeypatch.setattr(pacman_plugin.output, "print_summary", fake_print_summary)
ok = pacman.apply(store, dry_run=True)
assert ok is True
# to_remove = (native | orphans) - desired
# = {keep-explicit, old-explicit} {orphan-explicit} - {keep-explicit, new-pkg}
# = {old-explicit, orphan-explicit}
#
# old-explicit has foreign dependant -> demoted to dep
# orphan-explicit has no dependants -> removed
# printed lists (titles and contents)
titles = [t for t, _ in printed_lists]
assert "Removing pacman packages:" in titles
assert "Setting previously explicitly installed packages as dependencies:" in titles
assert "Installing pacman packages:" in titles
# find lists by title
remove_list = next(items for t, items in printed_lists if "Removing pacman packages:" in t)
demote_list = next(
items
for t, items in printed_lists
if "Setting previously explicitly installed packages as dependencies:" in t
)
install_list = next(items for t, items in printed_lists if "Installing pacman packages:" in t)
assert remove_list == ["orphan-explicit"]
assert demote_list == ["old-explicit"]
# to_install = desired - currently_installed_native
# = {keep-explicit, new-pkg} - {keep-explicit, old-explicit}
# = {new-pkg}
assert install_list == ["new-pkg"]
# Upgrade summary printed even in dry-run
assert any("Upgrading packages." in s for s in printed_summaries)
# No mutating calls in dry-run
assert fake_pm.remove_called_with is None
assert fake_pm.set_as_deps_called_with is None
assert fake_pm.upgrade_called is False
assert fake_pm.install_called_with is None
def test_apply_returns_false_on_command_failure(monkeypatch: pytest.MonkeyPatch) -> None:
pacman = pacman_plugin.Pacman()
store = FakeStore()
pacman.packages = set()
class FailingPM:
def __init__(self, *args, **kwargs) -> None: # noqa: D401
pass
def get_native_explicit(self) -> set[str]:
raise pacman_plugin.errors.CommandFailedError(["get_native_explicit"], 10, "boom")
monkeypatch.setattr(pacman_plugin, "PacmanInterface", FailingPM)
errors_logged: list[str] = []
continuations: list[str] = []
traceback_called = []
def fake_print_error(msg: str) -> None:
errors_logged.append(msg)
def fake_print_traceback() -> None:
traceback_called.append(True)
def fake_print_continuation(msg: str) -> None:
continuations.append(msg)
monkeypatch.setattr(pacman_plugin.output, "print_error", fake_print_error)
monkeypatch.setattr(pacman_plugin.output, "print_traceback", fake_print_traceback)
monkeypatch.setattr(pacman_plugin.output, "print_continuation", fake_print_continuation)
ok = pacman.apply(store, dry_run=False)
assert ok is False
assert any("Pacman command exited with an unexpected" in msg for msg in errors_logged)
assert any("boom" in msg for msg in continuations)
assert traceback_called # at least once
def test_ignored_packages_are_not_removed_or_installed(monkeypatch: pytest.MonkeyPatch) -> None:
pacman = pacman_plugin.Pacman()
store = FakeStore()
# Desired state: "already" and "new" should be managed normally.
# "ignored-installed" is currently installed but not desired -> would normally be removed.
# "ignored-uninstalled" is desired but not installed -> would normally be installed.
pacman.packages = {"already", "new", "ignored-uninstalled"}
pacman.ignored_packages = {"ignored-installed", "ignored-uninstalled"}
class FakePM:
def __init__(
self, commands, print_highlights, keywords, database_signature_level, database_path
) -> None: # noqa: D401
self.commands = commands
self.print_highlights = print_highlights
self.keywords = keywords
self.remove_called_with: set[str] | None = None
self.install_called_with: set[str] | None = None
self.set_as_deps_called_with: set[str] | None = None
self.upgrade_called = False
def get_native_explicit(self) -> set[str]:
# currently installed explicit packages
return {"ignored-installed", "already"}
def get_foreign_explicit(self) -> set[str]:
return set()
def get_native_orphans(self) -> set[str]:
return set()
def get_dependants(self, pkg: str) -> set[str]:
return set()
def remove(self, pkgs: set[str]) -> None:
self.remove_called_with = pkgs
def set_as_dependencies(self, pkgs: set[str]) -> None:
self.set_as_deps_called_with = pkgs
def upgrade(self) -> None:
self.upgrade_called = True
def install(self, pkgs: set[str]) -> None:
self.install_called_with = pkgs
fake_pm = FakePM(None, None, None, None, None)
def fake_pm_ctor(
commands, print_highlights, keywords, database_signature_level, database_path
) -> FakePM:
fake_pm.commands = commands
fake_pm.print_highlights = print_highlights
fake_pm.keywords = keywords
return fake_pm
monkeypatch.setattr(pacman_plugin, "PacmanInterface", fake_pm_ctor)
printed_lists: list[tuple[str, list[str]]] = []
def fake_print_list(title: str, items: list[str]) -> None:
printed_lists.append((title, items))
# don't care about summaries here
monkeypatch.setattr(pacman_plugin.output, "print_list", fake_print_list)
monkeypatch.setattr(pacman_plugin.output, "print_summary", lambda *_args, **_kw: None)
ok = pacman.apply(store, dry_run=False)
assert ok is True
# Ignored packages must never be passed to remove() or install()
assert (
fake_pm.remove_called_with is None or "ignored-installed" not in fake_pm.remove_called_with
)
assert fake_pm.install_called_with is not None
assert "ignored-uninstalled" not in fake_pm.install_called_with
# Also ensure the printed install list doesn't contain ignored packages
install_items = next(
items for title, items in printed_lists if "Installing pacman packages:" in title
)
assert "ignored-uninstalled" not in install_items
# "new" is the only package that should be installed in this scenario
assert install_items == ["new"]
@@ -0,0 +1,88 @@
import pyalpm
import pytest
from decman.plugins.aur import AurPacmanInterface
from decman.plugins.pacman import PacmanInterface
class FakePackage:
def __init__(self, name: str, is_explicit: bool, required_by: list[str]):
self.name = name
self.reason = pyalpm.PKG_REASON_EXPLICIT if is_explicit else pyalpm.PKG_REASON_DEPEND
self.required_by = required_by
self.provides = [name]
def compute_requiredby(self):
return self.required_by
class FakeDB:
def __init__(self, pkgcache: list[FakePackage]):
self.pkgcache = pkgcache
class FakePyalpmHandle:
def __init__(self):
pass
def get_syncdbs(self):
return [
FakeDB(
[
FakePackage("a", True, []),
FakePackage("b", False, ["a"]),
FakePackage("c", False, ["b"]),
FakePackage("d", False, []),
FakePackage("e", False, ["f"]),
FakePackage("f", False, ["g"]),
FakePackage("g", False, []),
]
)
]
def get_localdb(self):
return FakeDB(
self.get_syncdbs()[0].pkgcache
+ [
FakePackage("h", True, []),
FakePackage("i", False, ["h"]),
FakePackage("j", False, []),
FakePackage("k", False, ["l"]),
FakePackage("l", False, []),
]
)
def fake_create_pyalpm_handle(self):
return FakePyalpmHandle()
def test_get_native_orphans_pacman(
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.setattr(PacmanInterface, "_create_pyalpm_handle", fake_create_pyalpm_handle)
interface = PacmanInterface(
None, # type: ignore
False,
set(),
2048,
"/var/lib/pacman/",
)
assert interface.get_native_orphans() == {"d", "e", "f", "g"}
def test_get_foreign_orphans_aur(
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.setattr(AurPacmanInterface, "_create_pyalpm_handle", fake_create_pyalpm_handle)
interface = AurPacmanInterface(
None, # type: ignore
False,
set(),
2048,
"/var/lib/pacman/",
)
assert interface.get_foreign_orphans() == {"j", "k", "l"}
+383
View File
@@ -0,0 +1,383 @@
import typing
from unittest.mock import MagicMock
from urllib.parse import parse_qs, unquote, urlparse
import pytest
from decman.plugins.aur.commands import AurCommands
from decman.plugins.aur.fpm import ForeignPackageManager
from decman.plugins.aur.package import PackageInfo, PackageSearch
class FakeAurPacmanInterface:
def __init__(self) -> None:
self.installed_native: set[str] = set()
self.installed_foreign: dict[str, str] = {}
self.explicitly_installed: set[str] = set()
self.not_installable: set[str] = set()
self.installed_files: list[str] = [] # To track what install_files() actually does
self.provided_pkgs: set[str] = set()
def get_native_explicit(self) -> set[str]:
return self.installed_native.intersection(self.explicitly_installed)
def get_native_orphans(self) -> set[str]:
return set()
def get_foreign_explicit(self) -> set[str]:
return set(self.installed_foreign.keys()).intersection(self.explicitly_installed)
def get_dependants(self, package: str) -> set[str]:
return set()
def set_as_dependencies(self, packages: set[str]):
self.explicitly_installed.difference_update(packages)
def install(self, packages: set[str]):
self.installed_native.update(packages)
self.explicitly_installed.update(packages)
def upgrade(self):
pass
def is_provided_by_installed(self, dependency: str) -> bool:
return dependency in self.provided_pkgs
def get_all_packages(self) -> set[str]:
return self.installed_native | self.installed_foreign.keys()
def filter_installed_packages(self, deps: set[str]) -> set[str]:
out = set()
for d in deps:
if not self.is_provided_by_installed(d) and d not in self.get_all_packages():
out.add(d)
return out
def remove(self, packages: set[str]):
self.installed_native.difference_update(packages)
for p in packages:
self.installed_foreign.pop(p, None)
self.explicitly_installed.difference_update(packages)
def get_foreign_orphans(self) -> set[str]:
return set()
def is_installable(self, pkg: str) -> bool:
return pkg not in self.not_installable
def get_versioned_foreign_packages(self) -> list[tuple[str, str]]:
return list(self.installed_foreign.items())
def install_dependencies(self, deps: set[str]):
self.installed_native.update(deps)
def install_files(self, files: list[str], as_explicit: set[str]):
self.installed_files.extend(files)
for file in files:
self.installed_foreign[file] = "file"
for pkg in as_explicit:
self.explicitly_installed.add(pkg)
class FakeStore:
def __init__(self) -> None:
self._store: dict[str, typing.Any] = {}
def __getitem__(self, key: str) -> typing.Any:
return self._store[key]
def __setitem__(self, key: str, value: typing.Any) -> None:
self._store[key] = value
def get(self, key: str, default: typing.Any = None) -> typing.Any:
return self._store.get(key, default)
def ensure(self, key: str, default: typing.Any = None):
if key not in self._store:
self._store[key] = default
def __enter__(self) -> "FakeStore":
return self
def __exit__(self, exc_type, exc, tb):
return False
def save(self) -> None:
pass
def __repr__(self) -> str:
return repr(self._store)
class MockAurServer:
def __init__(self) -> None:
self.db: dict[str, dict] = {} # Maps pkgname -> raw JSON result dict
def seed(self, packages: list[PackageInfo]):
for pkg in packages:
# Reconstruct the raw JSON structure expected by PackageSearch
entry = {
"Name": pkg.pkgname,
"PackageBase": pkg.pkgbase or pkg.pkgname,
"Version": pkg.version,
"Description": "Mock Description",
"URL": "https://example.com",
"Depends": pkg.dependencies,
"MakeDepends": pkg.make_dependencies,
"CheckDepends": pkg.check_dependencies,
"Provides": pkg.provides,
# Add other fields if your class relies on them
}
self.db[pkg.pkgname] = entry
def handle_request(self, url, *args, **kwargs):
parsed = urlparse(url)
path = parsed.path
query = parse_qs(parsed.query)
results = []
# --- Handle: Multi-info query (.../info?arg[]=pkg1&arg[]=pkg2) ---
if "/rpc/v5/info" in path and "arg[]" in query:
requested_names = query["arg[]"]
for name in requested_names:
if name in self.db:
results.append(self.db[name])
# --- Handle: Single info query (.../rpc/v5/info/pkgname) ---
elif "/rpc/v5/info/" in path:
# Extract package name from end of path
pkg_name = path.split("/")[-1]
if pkg_name in self.db:
results.append(self.db[pkg_name])
# --- Handle: Search providers (.../rpc/v5/search/dep?by=provides) ---
elif "/rpc/v5/search/" in path and query.get("by") == ["provides"]:
search_term = path.split("/")[-1]
search_term = unquote(search_term)
# Linear search through DB for 'Provides'
for entry in self.db.values():
if search_term in entry.get("Provides", []):
results.append(entry)
# Also match if the package name itself matches the provider request
elif entry["Name"] == search_term:
results.append(entry)
# Construct the response object
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.json.return_value = {
"version": 5,
"type": "multiinfo",
"resultcount": len(results),
"results": results,
}
return mock_response
@pytest.fixture
def mock_aur(mocker):
server = MockAurServer()
mocker.patch("requests.get", side_effect=server.handle_request)
return server
@pytest.fixture
def mock_pacman(mocker):
pacman = FakeAurPacmanInterface()
return pacman
@pytest.fixture
def mock_fpm(mocker, mock_aur, mock_pacman):
mock_builder_cls = mocker.patch("decman.plugins.aur.fpm.PackageBuilder")
mock_builder_instance = mock_builder_cls.return_value
mock_builder_instance.__enter__.return_value = mock_builder_instance
mock_builder_instance.__exit__.return_value = None
# NOTE: find_latest_cached_package must return a tuple, otherwise
# the 'assert built_pkg is not None' line in install() will fail.
def mock_find_cached(store, package):
# return just the package, so that mock pacman can get the package name from the 'file' name
return ("1.0.0", package)
mocker.patch("decman.plugins.aur.fpm.find_latest_cached_package", side_effect=mock_find_cached)
mocker.patch("decman.plugins.aur.fpm.add_package_to_cache", return_value=None)
# handle prompts automatically
mocker.patch("decman.core.output.prompt_confirm", return_value=True)
store = FakeStore()
search = PackageSearch()
commands = AurCommands()
mgr = ForeignPackageManager(
store=store, # type: ignore
pacman=mock_pacman,
search=search,
commands=commands,
pkg_cache_dir="/tmp/cache",
build_dir="/tmp/build",
makepkg_user="nobody",
)
return mgr
def test_remove_pacman_deps_provided_by_foreign_packages(
mock_fpm, mock_aur, mock_pacman: FakeAurPacmanInterface
):
mock_pacman.not_installable |= {"kwin-hifps", "qt6-base-hifps", "syncthingtray-qt6"}
pkgs = [
PackageInfo(
pkgbase="kwin-hifps",
pkgname="kwin-hifps",
version="1",
git_url="...",
dependencies=("qt6-base-hifps",),
),
PackageInfo(
pkgbase="qt6-base-hifps",
pkgname="qt6-base-hifps",
version="1",
git_url="...",
provides=("qt6-base",),
),
PackageInfo(
pkgbase="syncthingtray-qt6",
pkgname="syncthingtray-qt6",
version="1",
git_url="...",
dependencies=("qt6-base",),
),
]
mock_aur.seed(pkgs)
mock_fpm.install(["kwin-hifps", "syncthingtray-qt6"])
assert len(mock_pacman.installed_files) == 3
assert mock_pacman.explicitly_installed == {"kwin-hifps", "syncthingtray-qt6"}
assert "qt6-base" not in mock_pacman.installed_native
def test_remove_pacman_deps_provided_by_already_installed_foreign_packages(
mock_fpm, mock_aur, mock_pacman: FakeAurPacmanInterface
):
mock_pacman.not_installable |= {"kwin-hifps", "qt6-base-hifps", "syncthingtray-qt6"}
pkgs = [
PackageInfo(
pkgbase="kwin-hifps",
pkgname="kwin-hifps",
version="1",
git_url="...",
dependencies=("qt6-base-hifps",),
),
PackageInfo(
pkgbase="qt6-base-hifps",
pkgname="qt6-base-hifps",
version="1",
git_url="...",
provides=("qt6-base",),
),
PackageInfo(
pkgbase="syncthingtray-qt6",
pkgname="syncthingtray-qt6",
version="1",
git_url="...",
dependencies=("qt6-base",),
),
]
mock_pacman.installed_foreign = {
"kwin-hifps": "1",
"qt6-base-hifps": "1",
}
mock_pacman.explicitly_installed.add("kwin-hifps")
mock_pacman.provided_pkgs.add("qt6-base")
mock_aur.seed(pkgs)
mock_fpm.install(["syncthingtray-qt6"])
assert len(mock_pacman.installed_files) == 1
assert mock_pacman.explicitly_installed == {"kwin-hifps", "syncthingtray-qt6"}
assert "qt6-base" not in mock_pacman.installed_native
def test_install_simple_package(
mock_fpm, mock_pacman: FakeAurPacmanInterface, mock_aur: MockAurServer
):
mock_pacman.not_installable.add("foo")
pkg = PackageInfo(
pkgbase="foo",
pkgname="foo",
version="100.0.0",
git_url="...",
)
mock_aur.seed([pkg])
mock_fpm.install(["foo"])
assert len(mock_pacman.installed_files) == 1
assert "foo" in mock_pacman.installed_files[0]
assert "foo" in mock_pacman.explicitly_installed
assert "foo" in mock_pacman.installed_foreign
def test_upgrade_foreign_package(mock_fpm, mock_pacman, mock_aur):
mock_pacman.not_installable.add("my-app")
mock_pacman.installed_foreign = {"my-app": "1.0"}
mock_pacman.explicitly_installed = {"my-app"}
pkg = PackageInfo(
pkgbase="my-app",
pkgname="my-app",
version="2.0",
git_url="...",
)
mock_aur.seed([pkg])
mock_fpm.upgrade()
assert len(mock_pacman.installed_files) == 1
assert "my-app" in mock_pacman.installed_foreign
assert "my-app" in mock_pacman.installed_files[0]
def test_upgrade_skips_current_package(mock_fpm, mock_pacman, mock_aur):
mock_pacman.not_installable.add("stable-app")
mock_pacman.installed_foreign = {"stable-app": "5.0"}
mock_pacman.explicitly_installed = {"stable-app"}
pkg = PackageInfo(
pkgbase="stable-app",
pkgname="stable-app",
version="5.0",
git_url="...",
)
mock_aur.seed([pkg])
mock_fpm.upgrade()
assert len(mock_pacman.installed_files) == 0
def test_install_resolves_dependencies(mock_fpm, mock_pacman, mock_aur):
mock_pacman.not_installable |= {"lib-helper", "main-app"}
pkg_dep = PackageInfo(pkgbase="lib-helper", pkgname="lib-helper", version="1.5", git_url="...")
pkg_main = PackageInfo(
pkgbase="main-app",
pkgname="main-app",
version="2.0",
dependencies=("lib-helper",),
git_url="...",
)
mock_aur.seed([pkg_dep, pkg_main])
mock_fpm.install(["main-app"])
assert len(mock_pacman.installed_files) == 2
assert "main-app" in mock_pacman.explicitly_installed
assert "main-app" in mock_pacman.installed_files
assert "lib-helper" in mock_pacman.installed_files
+20
View File
@@ -0,0 +1,20 @@
[project]
name = "decman-systemd"
version = "1.1.0"
requires-python = ">=3.13"
dependencies = ["decman==1.2.0"]
[project.entry-points."decman.plugins"]
systemd = "decman.plugins.systemd:Systemd"
[build-system]
requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"
[tool.setuptools.packages.find]
where = ["src"]
namespaces = true
include = ["decman.plugins*"]
[tool.pytest.ini_options]
testpaths = ["tests"]
@@ -0,0 +1,259 @@
import shutil
import decman.config as config
import decman.core.command as command
import decman.core.error as errors
import decman.core.module as module
import decman.core.output as output
import decman.core.store as _store
import decman.plugins as plugins
def units(fn):
"""
Annotate that this function returns a set of systemd unit names that should be enabled.
Return type of ``fn``: ``set[str]``
"""
fn.__systemd__units__ = True
return fn
def user_units(fn):
"""
Annotate that this function returns a dict of users and systemd user unit names that should be
enabled.
Return type of ``fn``: ``dict[str, set[str]]``
"""
fn.__systemd__user__units__ = True
return fn
class SystemdCommands:
"""
Default commands for the Systemd plugin.
"""
def enable_units(self, units: set[str]) -> list[str]:
"""
Running this command enables the given systemd units.
"""
return ["systemctl", "enable"] + list(units)
def disable_units(self, units: set[str]) -> list[str]:
"""
Running this command disables the given systemd units.
"""
return ["systemctl", "disable"] + list(units)
def enable_user_units(self, units: set[str], user: str) -> list[str]:
"""
Running this command enables the given systemd units for the user.
"""
return ["systemctl", "--user", "-M", f"{user}@", "enable"] + list(units)
def disable_user_units(self, units: set[str], user: str) -> list[str]:
"""
Running this command disables the given systemd units for the user.
"""
return ["systemctl", "--user", "-M", f"{user}@", "disable"] + list(units)
def daemon_reload(self) -> list[str]:
"""
Running this command reloads the systemd daemon.
"""
return ["systemctl", "daemon-reload"]
def user_daemon_reload(self, user: str) -> list[str]:
"""
Running this command reloads the systemd daemon for the given user.
"""
return ["systemctl", "--user", "-M", f"{user}@", "daemon-reload"]
class Systemd(plugins.Plugin):
NAME = "systemd"
def __init__(self) -> None:
self.enabled_units: set[str] = set()
self.enabled_user_units: dict[str, set[str]] = {}
self.commands = SystemdCommands()
def available(self) -> bool:
return shutil.which("systemctl") is not None
def process_modules(self, store: _store.Store, modules: list[module.Module]):
# These store keys are used to track changes in modules.
# This way when these change, module can be marked as changed
store.ensure("systemd_units_for_module", {})
store.ensure("systemd_user_units_for_module", {})
for mod in modules:
store["systemd_units_for_module"].setdefault(mod.name, set())
store["systemd_user_units_for_module"].setdefault(mod.name, {})
units = set().union(*plugins.run_methods_with_attribute(mod, "__systemd__units__"))
user_units = {
k: v
for d in plugins.run_methods_with_attribute(mod, "__systemd__user__units__")
for k, v in d.items()
}
if store["systemd_units_for_module"][mod.name] != units:
mod._changed = True
output.print_debug(
f"Module '{mod.name}' set to changed due to modified systemd units."
)
if store["systemd_user_units_for_module"][mod.name] != user_units:
mod._changed = True
output.print_debug(
f"Module '{mod.name}' set to changed due to modified systemd user units."
)
self.enabled_units |= units
for user, u_units in user_units.items():
self.enabled_user_units.setdefault(user, set()).update(u_units)
store["systemd_units_for_module"][mod.name] = units
store["systemd_user_units_for_module"][mod.name] = user_units
def apply(
self, store: _store.Store, dry_run: bool = False, params: list[str] | None = None
) -> bool:
store.ensure("systemd_units", set())
store.ensure("systemd_user_units", {})
units_to_enable = set()
units_to_disable = set()
user_units_to_enable: dict[str, set[str]] = {}
user_units_to_disable: dict[str, set[str]] = {}
for unit in self.enabled_units:
if unit not in store["systemd_units"]:
units_to_enable.add(unit)
for unit in store["systemd_units"]:
if unit not in self.enabled_units:
units_to_disable.add(unit)
for user, units in self.enabled_user_units.items():
store["systemd_user_units"].setdefault(user, set())
user_units_to_enable.setdefault(user, set())
for unit in units:
if unit not in store["systemd_user_units"][user]:
user_units_to_enable[user].add(unit)
for user, units in store["systemd_user_units"].items():
self.enabled_user_units.setdefault(user, set())
user_units_to_disable.setdefault(user, set())
for unit in units:
if unit not in self.enabled_user_units[user]:
user_units_to_disable[user].add(unit)
try:
output.print_list("Enabling systemd units:", list(units_to_enable))
if not dry_run:
self.enable_units(store, units_to_enable)
output.print_list("Disabling systemd units:", list(units_to_disable))
if not dry_run:
self.disable_units(store, units_to_disable)
for user, units in user_units_to_enable.items():
output.print_list(f"Enabling systemd units for {user}:", list(units))
if not dry_run:
self.enable_user_units(store, units, user)
for user, units in user_units_to_disable.items():
output.print_list(f"Disabling systemd units for {user}:", list(units))
if not dry_run:
self.disable_user_units(store, units, user)
output.print_info("Reloading systemd daemon.")
if not dry_run:
self.reload_daemon()
output.print_info("Reloading systemd daemon for users.")
if not dry_run:
for user in user_units_to_enable.keys() | user_units_to_disable.keys():
self.reload_user_daemon(user)
except errors.CommandFailedError as error:
output.print_error("Running a systemd command failed.")
output.print_error(str(error))
if error.output:
output.print_command_output(error.output)
output.print_traceback()
return False
return True
def enable_units(self, store: _store.Store, units: set[str]):
"""
Enables the given units.
"""
if not units:
return
cmd = self.commands.enable_units(units)
command.prg(cmd, pty=config.debug_output)
store["systemd_units"] |= units
def disable_units(self, store: _store.Store, units: set[str]):
"""
Disables the given units.
"""
if not units:
return
cmd = self.commands.disable_units(units)
command.prg(cmd, pty=config.debug_output, check=False)
store["systemd_units"] -= units
def enable_user_units(self, store: _store.Store, units: set[str], user: str):
"""
Enables the given units for the given user.
"""
if not units:
return
# Use check=False to avoid issues when units don't exist
cmd = self.commands.enable_user_units(units, user)
command.prg(cmd, pty=config.debug_output)
store["systemd_user_units"].setdefault(user, set())
store["systemd_user_units"][user] |= units
def disable_user_units(self, store: _store.Store, units: set[str], user: str):
"""
Disables the given units for the given user.
"""
if not units:
return
# Use check=False to avoid issues when units don't exist
cmd = self.commands.disable_user_units(units, user)
command.prg(cmd, pty=config.debug_output, check=False)
store["systemd_user_units"].setdefault(user, set())
store["systemd_user_units"][user] -= units
def reload_user_daemon(self, user: str):
"""
Reloads the user's systemd daemon.
"""
cmd = self.commands.user_daemon_reload(user)
command.prg(cmd, pty=config.debug_output)
def reload_daemon(self):
"""
Reloads the systemd daemon.
"""
cmd = self.commands.daemon_reload()
command.prg(cmd, pty=config.debug_output)
@@ -0,0 +1,297 @@
import pytest
from decman.plugins import systemd as systemd_mod
class DummyStore(dict):
def ensure(self, key, default):
if key not in self:
self[key] = default
class DummyModule:
def __init__(self, name: str):
self.name = name
self._changed = False
@pytest.fixture
def store():
return DummyStore()
@pytest.fixture
def systemd():
return systemd_mod.Systemd()
def test_units_decorator_sets_attribute():
@systemd_mod.units
def fn():
pass
assert getattr(fn, "__systemd__units__", False) is True
def test_user_units_decorator_sets_attribute():
@systemd_mod.user_units
def fn():
pass
assert getattr(fn, "__systemd__user__units__", False) is True
def test_available_true_if_systemctl_found(monkeypatch, systemd):
called = {}
def fake_which(name):
called["name"] = name
return "/bin/systemctl"
monkeypatch.setattr(systemd_mod.shutil, "which", fake_which)
assert systemd.available() is True
assert called["name"] == "systemctl"
def test_available_false_if_systemctl_missing(monkeypatch, systemd):
monkeypatch.setattr(systemd_mod.shutil, "which", lambda name: None)
assert systemd.available() is False
def test_process_modules_marks_changed_and_updates_store(monkeypatch, store, systemd):
# initial store empty; ensure keys will be created
m1 = DummyModule("mod1")
m2 = DummyModule("mod2")
def fake_run_method(mod, attr):
if mod is m1 and attr == "__systemd__units__":
return [{"a.service"}]
if mod is m1 and attr == "__systemd__user__units__":
return [{"alice": {"u1.service"}}]
# m2 has no units
return []
monkeypatch.setattr(systemd_mod.plugins, "run_methods_with_attribute", fake_run_method)
systemd.process_modules(store, {m1, m2})
# m1 changed from default -> marked _changed
assert m1._changed is True
# m2 had no units
assert m2._changed is False
# enabled units aggregated
assert systemd.enabled_units == {"a.service"}
assert systemd.enabled_user_units == {"alice": {"u1.service"}}
# store updated per module
assert store["systemd_units_for_module"]["mod1"] == {"a.service"}
assert store["systemd_user_units_for_module"]["mod1"] == {"alice": {"u1.service"}}
assert store["systemd_units_for_module"]["mod2"] == set()
assert store["systemd_user_units_for_module"]["mod2"] == {}
def test_process_modules_no_change_second_run(monkeypatch, store, systemd):
m1 = DummyModule("mod1")
def fake_run_method(mod, attr):
if attr == "__systemd__units__":
return [{"a.service"}]
if attr == "__systemd__user__units__":
return [{"alice": {"u1.service"}}]
return []
monkeypatch.setattr(systemd_mod.plugins, "run_methods_with_attribute", fake_run_method)
# first run populates store
systemd.process_modules(store, {m1})
m1._changed = False
# new instance (fresh per-process in real usage)
systemd2 = systemd_mod.Systemd()
monkeypatch.setattr(systemd_mod.plugins, "run_methods_with_attribute", fake_run_method)
systemd2.process_modules(store, {m1})
# values in store are same -> _changed stays False
assert m1._changed is False
def test_apply_enables_and_disables_units_and_user_units(store):
s = systemd_mod.Systemd()
# Current enabled according to modules
s.enabled_units = {"new.service"}
s.enabled_user_units = {"alice": {"newuser.service"}}
# Store says we had an old unit enabled before
store["systemd_units"] = {"old.service"}
store["systemd_user_units"] = {"alice": {"olduser.service"}}
calls = []
def fake_reload_daemon():
calls.append(("reload_daemon",))
def fake_reload_user_daemon(user):
calls.append(("reload_user_daemon", user))
def fake_enable_units(store_arg, units_arg):
calls.append(("enable_units", frozenset(units_arg)))
store_arg["systemd_units"] |= units_arg
def fake_disable_units(store_arg, units_arg):
calls.append(("disable_units", frozenset(units_arg)))
store_arg["systemd_units"] -= units_arg
def fake_enable_user_units(store_arg, units_arg, user):
calls.append(("enable_user_units", user, frozenset(units_arg)))
store_arg["systemd_user_units"].setdefault(user, set()).update(units_arg)
def fake_disable_user_units(store_arg, units_arg, user):
calls.append(("disable_user_units", user, frozenset(units_arg)))
store_arg["systemd_user_units"].setdefault(user, set()).difference_update(units_arg)
# patch instance methods (no self parameter expected)
s.reload_daemon = fake_reload_daemon
s.reload_user_daemon = fake_reload_user_daemon
s.enable_units = fake_enable_units
s.disable_units = fake_disable_units
s.enable_user_units = fake_enable_user_units
s.disable_user_units = fake_disable_user_units
result = s.apply(store, dry_run=False, params=None)
# reloads called once
assert ("reload_daemon",) in calls
assert ("reload_user_daemon", "alice") in calls
# enable/disable correct units
assert ("enable_units", frozenset({"new.service"})) in calls
assert ("disable_units", frozenset({"old.service"})) in calls
assert ("enable_user_units", "alice", frozenset({"newuser.service"})) in calls
assert ("disable_user_units", "alice", frozenset({"olduser.service"})) in calls
# store reconciled
assert store["systemd_units"] == {"new.service"}
assert store["systemd_user_units"]["alice"] == {"newuser.service"}
def test_apply_dry_run_does_not_mutate_store_or_call_commands(store):
s = systemd_mod.Systemd()
s.enabled_units = {"new.service"}
s.enabled_user_units = {"alice": {"newuser.service"}}
store["systemd_units"] = {"old.service"}
store["systemd_user_units"] = {"alice": {"olduser.service"}}
called = {"reload": False, "enable": False, "disable": False}
s.reload_daemon = lambda: called.__setitem__("reload", True) or True
s.reload_user_daemon = lambda user: called.__setitem__("reload", True) or True
s.enable_units = lambda st, u: called.__setitem__("enable", True) or True
s.disable_units = lambda st, u: called.__setitem__("disable", True) or True
s.enable_user_units = lambda st, u, user: called.__setitem__("enable", True) or True
s.disable_user_units = lambda st, u, user: called.__setitem__("disable", True) or True
result = s.apply(store, dry_run=True, params=None)
assert result is True
# no commands should be called
assert called == {"reload": False, "enable": False, "disable": False}
# store unchanged
assert store["systemd_units"] == {"old.service"}
assert store["systemd_user_units"]["alice"] == {"olduser.service"}
def test_enable_units_success(monkeypatch, store, systemd):
store["systemd_units"] = {"old.service"}
def fake_run(cmd, **kwargs):
assert cmd[0] == "systemctl"
assert cmd[1] == "enable"
assert "new.service" in cmd[2:]
return 0, "ok"
monkeypatch.setattr(systemd_mod.command, "run", fake_run)
systemd.enable_units(store, {"new.service"})
assert store["systemd_units"] == {"old.service", "new.service"}
def test_disable_units_success(monkeypatch, store, systemd):
store["systemd_units"] = {"old.service", "new.service"}
def fake_run(cmd, **kwargs):
assert cmd[0] == "systemctl"
assert cmd[1] == "disable"
assert "new.service" in cmd[2:]
return 0, "ok"
monkeypatch.setattr(systemd_mod.command, "run", fake_run)
systemd.disable_units(store, {"new.service"})
assert store["systemd_units"] == {"old.service"}
def test_enable_user_units_success(monkeypatch, store, systemd):
store["systemd_user_units"] = {"alice": {"olduser.service"}}
def fake_run(cmd, **kwargs):
assert cmd[0] == "systemctl"
assert "--user" in cmd
assert "enable" in cmd
assert "newuser.service" in cmd
return 0, "ok"
monkeypatch.setattr(systemd_mod.command, "run", fake_run)
systemd.enable_user_units(store, {"newuser.service"}, "alice")
assert store["systemd_user_units"]["alice"] == {
"olduser.service",
"newuser.service",
}
def test_disable_user_units_success(monkeypatch, store, systemd):
store["systemd_user_units"] = {"alice": {"olduser.service", "newuser.service"}}
def fake_run(cmd, **kwargs):
assert cmd[0] == "systemctl"
assert "--user" in cmd
assert "disable" in cmd
assert "newuser.service" in cmd
return 0, "ok"
monkeypatch.setattr(systemd_mod.command, "run", fake_run)
systemd.disable_user_units(store, {"newuser.service"}, "alice")
assert store["systemd_user_units"]["alice"] == {"olduser.service"}
def test_reload_daemon_uses_command_run(monkeypatch, systemd):
called = {}
def fake_run(cmd, **kwargs):
called["cmd"] = cmd
return 0, "ok"
monkeypatch.setattr(systemd_mod.command, "run", fake_run)
systemd.reload_daemon()
assert called["cmd"][:2] == ["systemctl", "daemon-reload"]
def test_reload_user_daemon_uses_command_run(monkeypatch, systemd):
called = {}
def fake_run(cmd, **kwargs):
called["cmd"] = cmd
return 0, "ok"
monkeypatch.setattr(systemd_mod.command, "run", fake_run)
systemd.reload_user_daemon("alice")
cmd = called["cmd"]
assert cmd[0] == "systemctl"
assert "--user" in cmd
assert "daemon-reload" in cmd
+55 -12
View File
@@ -1,18 +1,61 @@
[project]
name = "decman"
version = "1.2.0"
description = "Declarative package & configuration manager for Arch Linux."
license = "GPL-3.0-or-later"
license-files = ["LICENSE"]
authors = [
{name = "Kivi Kaitaniemi"}
]
requires-python = ">=3.13"
[project.optional-dependencies]
pacman = ["decman-pacman"]
systemd = ["decman-systemd"]
flatpak = ["decman-flatpak"]
[project.scripts]
decman = "decman.app:main"
[dependency-groups]
dev = [
"ruff>=0.14.9",
"pytest>=8.4.2",
]
[build-system]
requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"
[project]
name = "decman"
version = "0.2.1"
description = "Declarative package & configuration manager for Arch Linux."
license = {file = "LICENSE"}
authors = [
{name = "Kivi Kaitaniemi"}
]
dependencies = [
"requests"
[tool.uv.workspace]
members = [
"plugins/decman-pacman",
"plugins/decman-systemd",
"plugins/decman-flatpak",
]
[project.scripts]
decman = "decman.app:main"
[tool.uv.sources]
decman = { workspace = true }
decman-pacman = { workspace = true }
decman-systemd = { workspace = true }
decman-flatpak = { workspace = true }
[tool.pytest.ini_options]
testpaths = ["tests"]
[tool.ruff]
line-length = 100
target-version = "py313"
[tool.ruff.lint]
select = [
"E", "F", "W", # base style/errors
"I", # import sorting
"B", # bugbear
]
[tool.ruff.format]
quote-style = "double"
indent-style = "space"
skip-magic-trailing-comma = false
+105 -388
View File
@@ -1,403 +1,120 @@
"""
Module for writing system configurations for decman.
"""
import typing
import pwd
import grp
import shutil
import os
import subprocess
import decman.error
# Re-exports
from decman.core.command import prg
from decman.core.error import SourceError
from decman.core.fs import Directory, File, Symlink
from decman.core.module import Module
from decman.core.store import Store
from decman.plugins import Plugin, available_plugins
class UserRaisedError(Exception):
"""
Error raised by running source
"""
plugins: dict[str, Plugin] = available_plugins()
def __init__(self, message) -> None:
super().__init__(message)
def sh(sh_cmd: str,
user: typing.Optional[str] = None,
env_overrides: typing.Optional[dict[str, str]] = None):
"""
Shortcut for running a shell command.
"""
if env_overrides is None:
env_overrides = {}
env = os.environ.copy()
for var, val in env_overrides.items():
env[var] = val
if user is None:
# Quick access for default plugins
try:
subprocess.run(sh_cmd, shell=True, check=True, env=env)
except subprocess.CalledProcessError as e:
raise decman.error.UserFacingError(
f"Running user defined shell command '{sh_cmd}' failed."
) from e
else:
from decman.plugins.aur import AUR
from decman.plugins.pacman import Pacman
pacman: None | Pacman = None
_pacman = plugins.get("pacman", None)
if isinstance(_pacman, Pacman):
pacman = _pacman
aur: None | AUR = None
_aur = plugins.get("aur", None)
if isinstance(_aur, AUR):
aur = _aur
except ModuleNotFoundError:
pass
try:
uid = pwd.getpwnam(user).pw_uid
gid = pwd.getpwnam(user).pw_gid
except KeyError as e:
raise decman.error.UserFacingError(
f"Running user defined shell command failed because the user {user} doesn't exist."
) from e
from decman.plugins.flatpak import Flatpak
with subprocess.Popen(sh_cmd, shell=True, group=gid, user=uid,
env=env) as process:
if process.wait() != 0:
raise decman.error.UserFacingError(
f"Running user shell command '{sh_cmd}' as {user} failed.")
flatpak: None | Flatpak = None
_flatpak = plugins.get("flatpak", None)
if isinstance(_flatpak, Flatpak):
flatpak = _flatpak
except ModuleNotFoundError:
pass
def prg(command: list[str],
user: typing.Optional[str] = None,
env_overrides: typing.Optional[dict[str, str]] = None):
"""
Shortcut for running a program.
"""
if env_overrides is None:
env_overrides = {}
env = os.environ.copy()
for var, val in env_overrides.items():
env[var] = val
if user is None:
try:
subprocess.run(command, check=True, env=env)
except subprocess.CalledProcessError as e:
raise decman.error.UserFacingError(
f"Running user defined program '{command}' failed.") from e
else:
try:
uid = pwd.getpwnam(user).pw_uid
gid = pwd.getpwnam(user).pw_gid
except KeyError as e:
raise decman.error.UserFacingError(
f"Running user defined program failed because the user {user} doesn't exist."
) from e
from decman.plugins.systemd import Systemd
with subprocess.Popen(command, group=gid, user=uid,
env=env) as process:
if process.wait() != 0:
raise decman.error.UserFacingError(
f"Running user program '{command}' as {user} failed.")
systemd: None | Systemd = None
_systemd = plugins.get("systemd", None)
if isinstance(_systemd, Systemd):
systemd = _systemd
except ModuleNotFoundError:
pass
__all__ = [
"SourceError",
"File",
"Directory",
"Symlink",
"Module",
"Store",
"Plugin",
"prg",
"sh",
]
class File:
"""
A simple file that gets copied to the target.
"""
def __init__(
self,
source_file: typing.Optional[str] = None,
content: typing.Optional[str] = None,
bin_file: bool = False,
encoding: str = "utf-8",
owner: typing.Optional[str] = None,
group: typing.Optional[str] = None,
permissions: int = 0o644,
):
if source_file is None and content is None:
raise ValueError("Both source_file and content cannot be None.")
if source_file is not None and content is not None:
raise ValueError("Both source_file and content cannot be set.")
self.source_file = source_file
self.content = content
self.permissions = permissions
self.bin_file = bin_file
self.encoding = encoding
self.uid = None
self.gid = None
if owner is not None:
self.uid = pwd.getpwnam(owner).pw_uid
self.gid = pwd.getpwnam(owner).pw_gid
if group is not None:
self.gid = grp.getgrnam(group).gr_gid
def copy_to(self,
target: str,
variables: typing.Optional[dict[str, str]] = None):
"""
Copies the contents of this file to the target file.
"""
if variables is None:
variables = {}
target_directory = os.path.dirname(target)
def create_missing_dirs(dirct: str, uid: typing.Optional[int],
gid: typing.Optional[int]):
if not os.path.isdir(dirct):
parent_dir = os.path.dirname(dirct)
if not os.path.isdir(parent_dir):
create_missing_dirs(parent_dir, uid, gid)
os.mkdir(dirct)
if uid is not None:
assert gid is not None, "If uid is set, then gid is set."
os.chown(dirct, uid, gid)
create_missing_dirs(target_directory, self.uid, self.gid)
self._write_content(target, variables)
if self.uid is not None:
assert self.gid is not None, "If uid is set, then gid is set."
os.chown(target, self.uid, self.gid)
os.chmod(target, self.permissions)
def _write_content(self, target: str, variables: dict[str, str]):
if self.source_file is not None and (self.bin_file
or len(variables) == 0):
shutil.copy(self.source_file, target)
elif self.bin_file and self.content is not None:
with open(target, "wb") as file:
file.write(self.content.encode(encoding=self.encoding))
elif self.source_file is not None:
with open(self.source_file, "rt", encoding=self.encoding) as src:
content = src.read()
for var, value in variables.items():
content = content.replace(var, value)
with open(target, "wt", encoding=self.encoding) as file:
file.write(content)
else:
assert self.content is not None, "Content should be set since source_file was not set."
content = self.content
for var, value in variables.items():
content = content.replace(var, value)
with open(target, "wt", encoding=self.encoding) as file:
file.write(content)
class Directory:
"""
Contents of this directory will be copied to the target.
"""
def __init__(
self,
source_directory: str,
bin_files: bool = False,
encoding: str = "utf-8",
owner: typing.Optional[str] = None,
group: typing.Optional[str] = None,
permissions: int = 0o644,
):
self.source_directory = source_directory
self.bin_files = bin_files
self.encoding = encoding
self.permissions = permissions
self.owner = owner
self.group = group
self.uid = None
self.gid = None
if owner is not None:
self.uid = pwd.getpwnam(owner).pw_uid
self.gid = pwd.getpwnam(owner).pw_gid
if group is not None:
self.gid = grp.getgrnam(group).gr_gid
def copy_to(self,
target_directory: str,
variables: typing.Optional[dict[str, str]] = None,
only_print: bool = False) -> list[str]:
"""
Copies the files in this directory to the target directory.
Returns all created files.
"""
created = []
original_wd = os.getcwd()
try:
os.chdir(self.source_directory)
for src_dir, _, src_files in os.walk("."):
for src_file in src_files:
src_path = os.path.join(src_dir, src_file)
file = File(source_file=src_path,
bin_file=self.bin_files,
encoding=self.encoding,
owner=self.owner,
group=self.group,
permissions=self.permissions)
target = os.path.normpath(
os.path.join(target_directory, src_path))
created.append(target)
if not only_print:
file.copy_to(target, variables)
finally:
os.chdir(original_wd)
return created
class UserPackage:
"""
Defines a custom package.
"""
def __init__(
self,
pkgname: str,
version: str,
dependencies: list[str],
git_url: str,
pkgbase: typing.Optional[str] = None,
provides: typing.Optional[list[str]] = None,
make_dependencies: typing.Optional[list[str]] = None,
check_dependencies: typing.Optional[list[str]] = None,
):
if pkgbase is None:
pkgbase = pkgname
if provides is None:
provides = []
if make_dependencies is None:
make_dependencies = []
if check_dependencies is None:
check_dependencies = []
self.pkgname = pkgname
self.pkgbase = pkgbase
self.version = version
self.provides = provides
self.dependencies = dependencies
self.make_dependencies = make_dependencies
self.check_dependencies = check_dependencies
self.git_url = git_url
def __hash__(self) -> int:
return self.pkgname.__hash__()
def __eq__(self, value: object, /) -> bool:
if isinstance(value, self.__class__):
return value.pkgname == self.pkgname
return False
class Module:
"""
Collection of connected packages, services and files.
Inherit this class to create your own modules.
"""
def __init__(self, name: str, enabled: bool, version: str):
self.name = name
self.enabled = enabled
self.version = version
def on_enable(self):
"""
Override this method to run python code when this module gets enabled.
"""
def on_disable(self):
"""
Override this method to run python code when this module gets disabled.
Note! If this module is simply removed, the code will not exacute. Instead set enabled to
False.
"""
def after_update(self):
"""
Override this method to run python code after updating the system. If this module is
disabled, this will not run.
"""
def after_version_change(self):
"""
Override this method to run python code after the version of this module has changed.
"""
def files(self) -> dict[str, File]:
"""
Override this method to return files that should be installed as a part of this module.
"""
return {}
def directories(self) -> dict[str, Directory]:
"""
Override this method to return directories that should be installed as a part of this module.
"""
return {}
def file_variables(self) -> dict[str, str]:
"""
Override this method to return variables that should replaced with a new value inside
this module's text files.
"""
return {}
def pacman_packages(self) -> list[str]:
"""
Override this method to return pacman packages that should be installed as a part of this
Module.
"""
return []
def user_packages(self) -> list[UserPackage]:
"""
Override this method to return user packages that should be installed as a part of this
Module.
"""
return []
def aur_packages(self) -> list[str]:
"""
Override this method to return AUR packages that should be installed as a part of this
Module.
"""
return []
def systemd_units(self) -> list[str]:
"""
Override this method to return systemd units that should be enabled as a part of this
Module.
"""
return []
def systemd_user_units(self) -> dict[str, list[str]]:
"""
Override this method to return systemd user units that should be enabled as a part of this
Module.
"""
return {}
def __hash__(self) -> int:
return self.name.__hash__()
def __eq__(self, value: object, /) -> bool:
if isinstance(value, self.__class__):
return value.name == self.name
return False
packages: list[str] = []
aur_packages: list[str] = []
user_packages: list[UserPackage] = []
ignored_packages: list[str] = []
enabled_systemd_units: list[str] = []
enabled_systemd_user_units: dict[str, list[str]] = {}
# -----------------------------------------
# Global variables for system configuration
# -----------------------------------------
files: dict[str, File] = {}
directories: dict[str, Directory] = {}
symlinks: dict[str, str | Symlink] = {}
modules: list[Module] = []
execution_order: list[str] = [
"files",
"pacman",
"aur",
"systemd",
]
def sh(
sh_cmd: str,
user: typing.Optional[str] = None,
env_overrides: typing.Optional[dict[str, str]] = None,
mimic_login: bool = False,
pty: bool = True,
check: bool = True,
) -> str:
"""
Shortcut for running a shell command. Returns the output of that command.
Arguments:
sh_cmd:
Shell command to execute. The command is passed to the system shell /bin/sh.
user:
User name to run the command as. If set, the command is executed after dropping
privileges to this user.
env_overrides:
Environment variables to override or add for the command execution.
These values are merged on top of the current process environment.
mimic_login:
If mimic_login is True, will set the following environment variables according to the
given user's passwd file details. This only happens when user is set.
- HOME
- USER
- LOGNAME
- SHELL
pty:
If True, run the command inside a pseudo-terminal (PTY). This enables interactive
behavior and terminal-dependent programs. If False, run the command without a PTY
using standard subprocess execution.
check:
If True, raise CommandFailedError when the command exits with a non-zero status.
If False, print a warning when encountering a non-zero exit code.
"""
cmd = ["/bin/sh", "-c", sh_cmd]
return prg(
cmd, user=user, env_overrides=env_overrides, mimic_login=mimic_login, pty=pty, check=check
)
-3
View File
@@ -1,3 +0,0 @@
import decman.app
decman.app.main()
+258 -241
View File
@@ -1,17 +1,16 @@
"""
Module containing the CLI Application.
"""
import argparse
import os
import sys
import traceback
import decman
import decman.error as err
import decman.lib as l
import decman.config as conf
from decman.lib import fpm
import decman.core.error as errors
import decman.core.file_manager as file_manager
import decman.core.module as _module
import decman.core.output as output
import decman.core.store as _store
_STORE_FILE = "/var/lib/decman/store.json"
def main():
@@ -19,317 +18,335 @@ def main():
Main entry for the CLI app
"""
sys.pycache_prefix = os.path.join(conf.pkg_cache_dir, "python/")
sys.pycache_prefix = os.path.join(conf.cache_dir, "python/")
parser = argparse.ArgumentParser(
prog="decman",
description=
"Declarative package & configuration manager for Arch Linux",
epilog="See more help at: https://github.com/kiviktnm/decman")
description="Declarative package & configuration manager for Arch Linux",
epilog="See the documentation: https://github.com/kiviktnm/decman",
)
parser.add_argument("--source",
action="store",
help="python file containing configuration")
parser.add_argument("--source", action="store", help="python file containing configuration")
parser.add_argument(
"--print",
"--dry-run",
"--print",
action="store_true",
default=False,
help="print what would happen as a result of running decman")
parser.add_argument("--debug",
action="store_true",
default=False,
help="show debug output")
help="print what would happen as a result of running decman",
)
parser.add_argument("--debug", action="store_true", default=False, help="show debug output")
parser.add_argument(
"--no-packages",
action="store_true",
default=False,
help="don't upgrade any packages (including foreign packages)")
parser.add_argument("--no-foreign-packages",
action="store_true",
default=False,
help="don't upgrade foreign packages")
parser.add_argument("--no-files",
action="store_true",
default=False,
help="don't install any files")
parser.add_argument("--no-systemd-units",
action="store_true",
default=False,
help="don't enable/disable systemd units")
parser.add_argument("--no-commands",
action="store_true",
default=False,
help="don't run user specified commands")
parser.add_argument("--upgrade-devel",
action="store_true",
default=False,
help="upgrade devel packages")
"--skip", nargs="*", type=str, default=[], help="skip the following execution steps"
)
parser.add_argument(
"--force-build",
"--only", nargs="*", type=str, default=[], help="run only the following execution steps"
)
parser.add_argument(
"--no-hooks",
action="store_true",
default=False,
help="force building of packages that are already cached")
help="don't run hook methods for modules",
)
parser.add_argument(
"--no-color",
action="store_true",
default=False,
help="don't print messages with color",
)
parser.add_argument(
"--params", nargs="*", default=[], type=str, help="additional parameters passed to plugins"
)
args = parser.parse_args()
if not _is_root():
l.print_error("Not running as root. Please run decman as root.")
conf.debug_output = args.debug
if args.no_color:
conf.color_output = False
else:
conf.color_output = output.has_ansi_support()
if os.getuid() != 0:
output.print_error("Not running as root. Please run decman as root.")
sys.exit(1)
original_wd = os.getcwd()
failed = False
try:
store = l.Store.restore()
except err.UserFacingError as error:
l.print_error(error.user_facing_msg)
for line in traceback.format_exc().splitlines():
l.print_debug(line)
with _store.Store(_STORE_FILE, args.dry_run) as store:
try:
_execute_source(store, args)
failed = not run_decman(store, args)
except errors.SourceError as error:
output.print_error(f"Error raised manually in the source: {error}")
output.print_traceback()
failed = True
except errors.CommandFailedError as error:
output.print_error(str(error))
if error.output:
output.print_command_output(error.output)
output.print_traceback()
failed = True
except ValueError as error:
output.print_error("ValueError raised from the source.")
output.print_error(str(error))
output.print_traceback()
failed = True
except errors.InvalidOnDisableError as error:
output.print_error(str(error))
output.print_traceback()
failed = True
except Exception as error:
output.print_error(f"Unexpected error while running decman: {error}")
output.print_traceback()
failed = True
except OSError as error:
output.print_error(
f"Failed to access decman store file '{_STORE_FILE}': {error.strerror or str(error)}."
)
output.print_error("This may cause already completed operations to run again.")
output.print_traceback()
except KeyboardInterrupt:
output.print_error("Interrupted by the user.")
failed = True
finally:
os.chdir(original_wd)
if failed:
sys.exit(1)
errored = False
try:
opts = _set_up(store, args)
# Override debug_output if cli option is used
if args.debug:
conf.debug_output = True
conf.suppress_command_output = False
# When print cli option is used, show info output
if args.print:
conf.quiet_output = False
Core(store, opts).run()
except err.UserFacingError as error:
l.print_error(error.user_facing_msg)
for line in traceback.format_exc().splitlines():
l.print_debug(line)
errored = True
except decman.UserRaisedError as user_error:
l.print_error(
f"Error encountered while running the source: {user_error}")
errored = True
def _execute_source(store: _store.Store, args: argparse.Namespace):
"""
Runs decman source. May call ``sys.exit(1)`` if user aborts running the source or reading the
source fails.
# Save even when an error has occurred, since this avoids repeating steps like building pkgs.
try:
store.save()
except err.UserFacingError as error:
l.print_error(error.user_facing_msg)
for line in traceback.format_exc().splitlines():
l.print_debug(line)
errored = True
Raises:
``SourceError``
If code in the source raises this error manually.
os.chdir(original_wd)
if errored:
sys.exit(2)
def _set_up(store: l.Store, args):
source = store.source_file
``InvalidOnDisableError``
If modules in the source have invalid on_disable functions.
"""
source = store.get("source_file", None)
source_changed = False
if args.source is not None:
source = args.source
source_changed = True
if source is None:
l.print_error(
output.print_error(
"Source was not specified. Please specify a source with the '--source' argument."
)
l.print_info("Decman will remember the previously specified source.")
output.print_info("Decman will remember the previously specified source.")
sys.exit(1)
if source_changed or not store.allow_running_source_without_prompt:
l.print_warning(f"Decman will run the file '{source}' as root!")
l.print_warning(
if source_changed or not store.get("allow_running_source_without_prompt", False):
output.print_warning(f"Decman will run the file '{source}' as root!")
output.print_warning(
"Only proceed if you trust the file completely. The file can also import other files."
)
if not l.prompt_confirm("Proceed?", default=False):
if not output.prompt_confirm("Proceed?", default=False):
sys.exit(1)
if l.prompt_confirm("Remember this choice?", default=False):
store.allow_running_source_without_prompt = True
if output.prompt_confirm("Remember this choice?", default=False):
store["allow_running_source_without_prompt"] = True
source_path = os.path.abspath(source)
source_dir = os.path.dirname(source_path)
store.source_file = source_path
store["source_file"] = source_path
try:
with open(source_path, "rt", encoding="utf-8") as file:
content = file.read()
except OSError as e:
raise err.UserFacingError(
f"Failed to read source file '{store.source_file}'.") from e
except OSError as error:
output.print_error(f"Failed to read source '{source_path}': {error.strerror or str(error)}")
sys.exit(1)
os.chdir(source_dir)
sys.path.append(".")
exec(content)
return args.print, not args.no_packages, not args.no_foreign_packages, not args.no_files, not args.no_systemd_units, not args.no_commands, args.upgrade_devel, args.force_build
class Core:
def run_decman(store: _store.Store, args: argparse.Namespace) -> bool:
"""
Contains the main logic of decman.
Runs decman with the given arguments and a store.
Returns ``True`` if executed succesfully. Otherwise ``False``.
Raises:
``SourceError``
If code in the source raises this error manually.
``CommandFailedError``
If running any command fails.
"""
def __init__(self, store: l.Store, opts):
self.only_print, self.update_packages, self.update_foreign_packages, self.update_files, self.update_units, self.run_commands, self.upgrade_devel, self.force_build = opts
output.print_debug(f"Available plugins: {' '.join(decman.plugins.keys())}")
self.store = store
self.source = _resolve_source()
self.pacman = l.Pacman()
self.systemctl = l.Systemd(store)
self.fpkg_search = fpm.ExtendedPackageSearch(self.pacman)
store.ensure("enabled_modules", [])
store.ensure("module_on_disable_scripts", {})
for upkg in self.source.all_user_pkgs():
self.fpkg_search.add_user_pkg(
fpm.PackageInfo.from_user_package(upkg, self.pacman))
execution_order = _determine_execution_order(args)
new_modules = _find_new_modules(store)
disabled_modules = _find_disabled_modules(store)
self.fpm = fpm.ForeignPackageManager(store, self.pacman,
self.fpkg_search)
# Disable hooks should be run before anything else because they might depend on packages that
# are going to get removed.
if not args.no_hooks:
_run_before_update(store, args)
_run_on_disable(store, args, disabled_modules)
def run(self):
"""
Run the main logic of decman.
"""
if self.update_units:
self._disable_units()
# Run main execution order
for step in execution_order:
output.print_info(f"Running step '{step}'.")
match step:
case "files":
if not file_manager.update_files(
store,
decman.modules,
decman.files,
decman.directories,
decman.symlinks,
dry_run=args.dry_run,
):
return False
case plugin_name:
plugin = decman.plugins.get(plugin_name, None)
if plugin:
plugin.process_modules(store, decman.modules)
if not plugin.apply(store, dry_run=args.dry_run, params=args.params):
return False
else:
output.print_warning(
f"Plugin '{plugin_name}' configured in execution_order, "
"but not found in available plugins."
)
if self.update_files:
self._create_and_remove_files()
# On enable and on change should be ran last since they might depend on effects caused by
# execution steps.
if not args.no_hooks:
_run_on_enable(store, args, new_modules)
_run_on_change(store, args)
_run_after_update(store, args)
if self.update_packages:
self._remove_pkgs()
self._upgrade_pkgs()
self._install_pkgs()
return True
if self.update_units:
self._enable_units()
if self.run_commands:
self._run_modules()
all_enabled_modules = {}
for mod, version in self.source.all_enabled_modules():
all_enabled_modules[mod] = version
# Enabled modules are really only stored for commands,
# so they can be set only when the commands were exacuted.
self.store.enabled_modules = all_enabled_modules
def _determine_execution_order(args: argparse.Namespace) -> list[str]:
execution_order = []
def _disable_units(self):
to_disable = self.source.units_to_disable(self.store)
l.print_list("Disabling systemd units:", to_disable)
if not self.only_print:
self.systemctl.disable_units(to_disable)
if args.only:
output.print_debug("Argument '--only' is set. Pruning execution steps.")
for step in decman.execution_order:
if step in args.only:
output.print_debug(f"Adding {step} to execution order.")
execution_order.append(step)
else:
execution_order = decman.execution_order
user_units_to_disable = self.source.user_units_to_disable(self.store)
for user, units in user_units_to_disable.items():
l.print_list(f"Disabling systemd units for {user}:", units)
if not self.only_print:
self.systemctl.disable_user_units(units, user)
for skip in args.skip:
output.print_debug(f"Skipping step {skip}.")
execution_order.remove(skip)
def _remove_pkgs(self):
currently_installed = self.pacman.get_installed()
to_remove = self.source.packages_to_remove(currently_installed)
l.print_list("Removing packages:", to_remove)
if not self.only_print:
self.pacman.remove(to_remove)
output.print_debug(f"Execution order is: {', '.join(execution_order)}.")
return execution_order
def _upgrade_pkgs(self):
l.print_summary("Upgrading packages.")
if not self.only_print:
self.pacman.upgrade()
if conf.enable_fpm and self.update_foreign_packages:
self.fpm.upgrade(self.upgrade_devel, self.force_build,
self.source.ignored_packages)
def _install_pkgs(self):
currently_installed = self.pacman.get_installed()
to_install_pacman = self.source.pacman_packages_to_install(
currently_installed)
to_install_fpm = self.source.foreign_packages_to_install(
currently_installed)
def _find_new_modules(store: _store.Store):
new_modules = []
for module in decman.modules:
if module.name not in store["enabled_modules"]:
new_modules.append(module.name)
output.print_debug(f"New modules are: {', '.join(new_modules)}.")
return new_modules
l.print_list("Installing pacman packages:", to_install_pacman)
# fpm prints a summary so no need to print it twice
if self.only_print:
l.print_list("Installing foreign packages:", to_install_fpm)
def _find_disabled_modules(store: _store.Store):
disabled_modules = []
enabled_module_names = set(map(lambda m: m.name, decman.modules))
for module_name in store["enabled_modules"]:
if module_name not in enabled_module_names:
disabled_modules.append(module_name)
output.print_debug(f"Disabled modules are: {', '.join(disabled_modules)}.")
return disabled_modules
if not self.only_print:
self.pacman.install(to_install_pacman)
if conf.enable_fpm and self.update_foreign_packages:
self.fpm.install(to_install_fpm, force=self.force_build)
def _create_and_remove_files(self):
l.print_summary("Installing files.")
def _run_before_update(store: _store.Store, args: argparse.Namespace):
output.print_summary("Running before_update -hooks.")
for module in decman.modules:
output.print_debug(f"Running before_update for {module.name}.")
if not args.dry_run:
module.before_update(store)
all_created = self.source.create_all_files(self.only_print)
to_remove = self.source.files_to_remove(self.store, all_created)
l.print_list("Ensured files are up to date:",
all_created,
elements_per_line=1)
l.print_list("Removing files:", to_remove, elements_per_line=1)
if self.only_print:
def _run_on_disable(store: _store.Store, args: argparse.Namespace, disabled_modules: list[str]):
if not disabled_modules:
return
for file in to_remove:
output.print_summary("Running on_disable -scripts.")
for disabled_module in disabled_modules:
on_disable_script = store["module_on_disable_scripts"].get(disabled_module, None)
if on_disable_script:
output.print_debug(f"Running on_disable for {disabled_module}.")
if not args.dry_run:
decman.prg([on_disable_script])
store["enabled_modules"].remove(disabled_module)
store["module_on_disable_scripts"].pop(disabled_module)
def _run_on_enable(store: _store.Store, args: argparse.Namespace, new_modules: list[str]):
if not new_modules:
return
output.print_summary("Running on_enable -hooks.")
for module in decman.modules:
if module.name in new_modules:
output.print_debug(f"Running on_enable for {module.name}.")
if not args.dry_run:
module.on_enable(store)
store["enabled_modules"].append(module.name)
try:
os.remove(file)
except OSError as e:
l.print_error(f"{e}")
l.print_warning(f"Failed to remove file: {file}")
self.store.created_files = all_created
def _enable_units(self):
to_enable = self.source.units_to_enable(self.store)
l.print_list("Enabling systemd units:", to_enable)
if not self.only_print:
self.systemctl.enable_units(to_enable)
user_units_to_enable = self.source.user_units_to_enable(self.store)
for user, units in user_units_to_enable.items():
l.print_list(f"Enabling systemd units for {user}:", units)
if not self.only_print:
self.systemctl.enable_user_units(units, user)
def _run_modules(self):
l.print_summary("Running on enable hooks.")
if not self.only_print:
self.source.run_on_enable(self.store)
l.print_summary("Running after version change hooks.")
if not self.only_print:
self.source.run_after_version_change(self.store)
l.print_summary("Running on disable hooks.")
if not self.only_print:
self.source.run_on_disable(self.store)
l.print_summary("Running after update hooks.")
if not self.only_print:
self.source.run_after_update()
def _resolve_source() -> l.Source:
enabled_systemd_user_units = {}
for user, units in decman.enabled_systemd_user_units.items():
enabled_systemd_user_units[user] = set(units)
return l.Source(
pacman_packages=set(decman.packages),
aur_packages=set(decman.aur_packages),
user_packages=set(decman.user_packages),
ignored_packages=set(decman.ignored_packages),
systemd_units=set(decman.enabled_systemd_units),
systemd_user_units=enabled_systemd_user_units,
files=decman.files,
directories=decman.directories,
modules=set(decman.modules),
script = _module.write_on_disable_script(
module, conf.module_on_disable_scripts_dir
)
if script:
store["module_on_disable_scripts"][module.name] = script
except OSError as error:
output.print_error(
f"Failed to create on_disable script for module {module.name}: "
)
output.print_error(f"{error.strerror or str(error)}.")
output.print_traceback()
output.print_warning(
"This script will NOT be created when decman runs the next time."
)
output.print_warning(
"You should investigate the reason for the error and try to fix it."
)
output.print_warning(
"Then disable and re-enable this module to create the script."
)
def _is_root() -> bool:
return os.geteuid() == 0
def _run_on_change(store: _store.Store, args: argparse.Namespace):
output.print_summary("Running on_change -hooks.")
for module in decman.modules:
if module._changed:
output.print_debug(f"Running on_change for {module.name}.")
if not args.dry_run:
module.on_change(store)
def _run_after_update(store: _store.Store, args: argparse.Namespace):
output.print_summary("Running after_update -hooks.")
for module in decman.modules:
output.print_debug(f"Running after_update for {module.name}.")
if not args.dry_run:
module.after_update(store)
+4 -181
View File
@@ -20,188 +20,11 @@ To change the defalts, create a new child class of the Commands-class and set th
variable to an instance of your class. Look in the example directory for an example.
"""
import typing
class Commands:
"""
Default commands.
"""
def list_pkgs(self) -> list[str]:
"""
Running this command outputs a newline seperated list of explicitly installed packages.
"""
return ["pacman", "-Qeq", "--color=never"]
def list_foreign_pkgs_versioned(self) -> list[str]:
"""
Running this command outputs a newline seperated list of installed packages and their
versions that are not from pacman repositories.
"""
return ["pacman", "-Qm", "--color=never"]
def install_pkgs(self, pkgs: list[str]) -> list[str]:
"""
Running this command installs the given packages from pacman repositories.
"""
return ["pacman", "-S", "--needed"] + pkgs
def install_files(self, pkg_files: list[str]) -> list[str]:
"""
Running this command installs the given packages files.
"""
return ["pacman", "-U", "--asdeps"] + pkg_files
def set_as_explicitly_installed(self, pkgs: list[str]) -> list[str]:
"""
Running this command installs sets the given as explicitly installed.
"""
return ["pacman", "-D", "--asexplicit"] + pkgs
def install_deps(self, deps: list[str]) -> list[str]:
"""
Running this command installs the given packages from pacman repositories.
The packages are installed as dependencies.
"""
return ["pacman", "-S", "--needed", "--asdeps"] + deps
def is_installable(self, pkg: str) -> list[str]:
"""
This command exits with code 0 when a package is installable from pacman repositories.
"""
return ["pacman", "-Sddp", pkg]
def upgrade(self) -> list[str]:
"""
Running this command upgrades all pacman packages.
"""
return ["pacman", "-Syu"]
def remove(self, pkgs: list[str]) -> list[str]:
"""
Running this command removes the given packages and their dependencies
(that aren't required by other packages).
"""
return ["pacman", "-Rs"] + pkgs
def enable_units(self, units: list[str]) -> list[str]:
"""
Running this command enables the given systemd units.
"""
return ["systemctl", "enable"] + units
def disable_units(self, units: list[str]) -> list[str]:
"""
Running this command disables the given systemd units.
"""
return ["systemctl", "disable"] + units
def enable_user_units(self, units: list[str], user: str) -> list[str]:
"""
Running this command enables the given systemd units for the user.
"""
return ["systemctl", "--user", "-M", f"{user}@", "enable"] + units
def disable_user_units(self, units: list[str], user: str) -> list[str]:
"""
Running this command disables the given systemd units for the user.
"""
return ["systemctl", "--user", "-M", f"{user}@", "disable"] + units
def compare_versions(self, installed_version: str,
new_version: str) -> list[str]:
"""
Running this command outputs -1 when the installed version is older than the new version.
"""
return ["vercmp", installed_version, new_version]
def git_clone(self, repo: str, dest: str) -> list[str]:
"""
Running this command clones a git repository to the the given destination.
"""
return ["git", "clone", repo, dest]
def git_diff(self, from_commit: str) -> list[str]:
"""
Running this command outputs the difference between the given commit and
the current state of the repository.
"""
return ["git", "diff", from_commit]
def git_get_commit_id(self) -> list[str]:
"""
Running this command outputs the current commit id.
"""
return ["git", "rev-parse", "HEAD"]
def review_file(self, file: str) -> list[str]:
"""
Running this command outputs a file for the user to see.
"""
return ["less", file]
def make_chroot(self, chroot_dir: str, with_pkgs: list[str]) -> list[str]:
"""
Running this command creates a new arch chroot to the chroot directory and installs the
given packages there.
"""
return ["mkarchroot", chroot_dir] + with_pkgs
def install_chroot_packages(self, chroot_dir: str, packages: list[str]):
"""
Running this command installs the given packages to the given chroot.
"""
return [
"arch-nspawn", chroot_dir, "pacman", "-S", "--needed",
"--noconfirm"
] + packages
def remove_chroot_packages(self, chroot_dir: str, packages: list[str]):
"""
Running this command removes the given packages from the given chroot.
"""
return ["arch-nspawn", chroot_dir, "pacman", "-Rsu", "--noconfirm"
] + packages
def make_chroot_pkg(self, chroot_wd_dir: str, user: str,
pkgfiles_to_install: list[str]) -> list[str]:
"""
Running this command creates a package file using the given chroot.
The package is created as the user and the pkg_files_to_install are installed
in the chroot before the package is created.
"""
makechrootpkg_cmd = [
"makechrootpkg", "-c", "-r", chroot_wd_dir, "-U", user
]
for pkgfile in pkgfiles_to_install:
makechrootpkg_cmd += ["-I", pkgfile]
return makechrootpkg_cmd
commands: Commands = Commands()
debug_output: bool = False
quiet_output: bool = False
suppress_command_output: bool = True
color_output: bool = True
valid_pkgexts: list[str] = [
".pkg.tar",
".pkg.tar.gz",
".pkg.tar.bz2",
".pkg.tar.xz",
".pkg.tar.zst",
".pkg.tar.lzo",
".pkg.tar.lrz",
".pkg.tar.lz4",
".pkg.tar.lz",
".pkg.tar.Z",
]
module_on_disable_scripts_dir: str = "/var/lib/decman/scripts/"
cache_dir: str = "/var/cache/decman"
makepkg_user: str = "nobody"
build_dir: str = "/tmp/decman/build"
pkg_cache_dir: str = "/var/cache/decman"
aur_rpc_timeout: typing.Optional[int] = 30
enable_fpm: bool = True
number_of_packages_stored_in_cache: int = 3
arch: str = "x86_64"
View File
+360
View File
@@ -0,0 +1,360 @@
import errno
import fcntl
import os
import pty
import pwd
import select
import shlex
import shutil
import signal
import struct
import subprocess
import sys
import termios
import tty
import typing
import decman.core.error as errors
import decman.core.output as output
def get_user_info(user: str) -> tuple[int, int]:
"""
Returns UID and GID of the given user.
If the user doesn't exist, raises UserNotFoundError.
"""
info = _get_passwd(user)
return info.pw_uid, info.pw_gid
def prg(
cmd: list[str],
user: typing.Optional[str] = None,
env_overrides: typing.Optional[dict[str, str]] = None,
pass_environment: bool = True,
mimic_login: bool = False,
pty: bool = True,
check: bool = True,
) -> str:
"""
Shortcut for running a command. Returns the output of that command.
Arguments:
cmd:
Command to execute.
user:
User name to run the command as. If set, the command is executed after dropping
privileges to this user.
env_overrides:
Environment variables to override or add for the command execution.
These values are merged on top of the current process environment.
mimic_login:
If mimic_login is True, will set the following environment variables according to the
given user's passwd file details. This only happens when user is set.
- HOME
- USER
- LOGNAME
- SHELL
pty:
If True, run the command inside a pseudo-terminal (PTY). This enables interactive
behavior and terminal-dependent programs. If False, run the command without a PTY
using standard subprocess execution.
If running in a PTY, the raised CommandFailedError will not contain command output,
since it has already been shown to the user.
check:
If True, raise CommandFailedError when the command exits with a non-zero status.
If False, print a warning when encountering a non-zero exit code.
"""
if pty:
result = pty_run(
cmd,
user=user,
env_overrides=env_overrides,
pass_environment=pass_environment,
mimic_login=mimic_login,
)
else:
result = run(
cmd,
user=user,
env_overrides=env_overrides,
pass_environment=pass_environment,
mimic_login=mimic_login,
)
if check:
# This raises an error if the command failed exiting the function early
result = check_run_result(cmd, result, include_output=not pty)
code, command_output = result
if code != 0:
output.print_warning(f"Command '{shlex.join(cmd)}' returned with an exit code {code}.")
if not pty:
output.print_command_output(command_output)
return command_output
def pty_run(
command: list[str],
user: None | str = None,
env_overrides: None | dict[str, str] = None,
mimic_login: bool = False,
pass_environment: bool = True,
) -> tuple[int, str]:
"""
Runs a given command with the given arguments in a pseudo TTY. The command can be ran as
the given user and environment variables can be overridden manually.
By default this will copy the current environment and pass it to the process. To prevent this
set ``pass_environment`` to ``False``.
If ``mimic_login`` is True, will set the following environment variables according to the given
user's passwd file details. This only happens when user is set.
- HOME
- USER
- LOGNAME
- SHELL
If the given command is empty, returns (0, "").
Returns the return code of the command and the output as a string.
If the user doesn't exist, raises UserNotFoundError.
If forking the process fails or stdin is not a TTY, raises OSError.
"""
if not command:
return 0, ""
if not sys.stdin.isatty():
raise OSError(errno.ENOTTY, "Stdin is not a TTY.")
command[0] = shutil.which(command[0]) or command[0]
output.print_debug(f"Running command '{shlex.join(command)}'.")
env = _build_env(user, env_overrides, mimic_login, pass_environment)
pid, master_fd = pty.fork()
if pid == 0:
_exec_in_child(command, env, user)
return _run_parent(master_fd, pid)
def run(
command: list[str],
user: None | str = None,
env_overrides: None | dict[str, str] = None,
mimic_login: bool = False,
pass_environment: bool = True,
) -> tuple[int, str]:
"""
Runs a given command with the given arguments. The command can be ran as the given user and
environment variables can be overridden manually.
By default this will copy the current environment and pass it to the process. To prevent this
set ``pass_environment`` to ``False``.
If mimic_login is True, will set the following environment variables according to the given
user's passwd file details. This only happens when user is set.
- HOME
- USER
- LOGNAME
- SHELL
If the given command is empty, returns (0, "").
Returns the return code of the command and the output as a string.
If the user doesn't exist, raises UserNotFoundError.
"""
if not command:
return 0, ""
command[0] = shutil.which(command[0]) or command[0]
output.print_debug(f"Running command '{shlex.join(command)}'.")
env = _build_env(user, env_overrides, mimic_login, pass_environment)
uid, gid = None, None
if user:
uid, gid = get_user_info(user)
try:
process = subprocess.Popen(
command, env=env, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, user=uid, group=gid
)
stdout, _ = process.communicate()
except OSError as error:
# Mirror PTY behavior: "<cmd>: <error>\n" and errno-based exit code
msg = error.strerror or str(error)
text_output = f"{command[0]}: {msg}\n"
code = error.errno if error.errno and error.errno < 128 else 127
return code, text_output
return process.returncode, stdout.decode("utf-8", errors="replace")
def check_run_result(
command: list[str], result: tuple[int, str], include_output: bool = True
) -> tuple[int, str]:
"""
Validates the result of a command execution.
If the command exited with a non-zero return code, raises CommandFailedError
containing the original command and its captured output.
Otherwise, returns the result unchanged.
"""
code, output = result
if code != 0:
if include_output:
raise errors.CommandFailedError(command, code, output)
else:
raise errors.CommandFailedError(command, code, None)
return code, output
def _build_env(
user: None | str,
env_overrides: None | dict[str, str],
mimic_login: bool,
pass_environment: bool,
) -> dict[str, str]:
env = {}
cwd = os.getcwd()
output.print_debug(
f"Command environment is: cwd='{cwd}', user='{user}', env_overrides='{env_overrides}', "
f"mimic_login='{mimic_login}', pass_environment='{pass_environment}'"
)
if pass_environment:
env = os.environ.copy()
if mimic_login and user:
pw = _get_passwd(user)
env.update(
{
"HOME": pw.pw_dir,
"USER": pw.pw_name,
"LOGNAME": pw.pw_name,
"SHELL": pw.pw_shell,
}
)
if env_overrides:
env.update(env_overrides)
return env
def _exec_in_child(command: list[str], env: dict[str, str], user: None | str) -> typing.NoReturn:
try:
if user:
uid, gid = get_user_info(user=user)
os.setgid(gid)
os.setuid(uid)
os.execve(command[0], command, env)
except OSError as error:
try:
os.write(2, f"{command[0]}: {error.strerror}\n".encode())
except OSError:
# Not much can be done, if outputting the failure state fails
pass
code = error.errno if (error.errno and error.errno < 128) else 127
os._exit(code)
def _run_parent(master_fd: int, pid: int) -> tuple[int, str]:
stdin_fd = sys.stdin.fileno()
stdout_fd = sys.stdout.fileno()
# Put stdin into raw mode and save previous termios attributes.
old_tattr = termios.tcgetattr(stdin_fd)
tty.setraw(stdin_fd)
# Helper function to set PTY window size to the current terminal size
def resize_pty(*args):
try:
cols, rows = shutil.get_terminal_size()
winsz = struct.pack("HHHH", rows, cols, 0, 0)
fcntl.ioctl(master_fd, termios.TIOCSWINSZ, winsz)
except OSError:
# In case the child has exited before the signal handled was de-registered
pass
# Set PTY window size to match the current terminal size.
resize_pty()
# Handle terminal resizes automatically
old_winch = signal.getsignal(signal.SIGWINCH)
signal.signal(signal.SIGWINCH, resize_pty)
try:
output_bytes = _relay_pty(master_fd, stdin_fd, stdout_fd)
finally:
# Restore stdin termios attributes.
termios.tcsetattr(stdin_fd, termios.TCSADRAIN, old_tattr)
# Restore previous handler
signal.signal(signal.SIGWINCH, old_winch)
os.close(master_fd)
_, status = os.waitpid(pid, 0)
exitcode = os.waitstatus_to_exitcode(status)
output = output_bytes.decode("utf-8", errors="replace").replace("\r\n", "\n")
return exitcode, output
def _relay_pty(master_fd: int, stdin_fd: int, stdout_fd: int) -> bytes:
"""
Drive interactive I/O between stdin/stdout and the PTY, capturing output.
"""
output_chunks: list[bytes] = []
while True:
# Wait until process or stdin has data
rlist, _, _ = select.select([master_fd, stdin_fd], [], [])
# Capture and echo child process
if master_fd in rlist:
try:
data = os.read(master_fd, 1024)
except OSError:
# Child process probably exited, EOF
break
output_chunks.append(data)
try:
os.write(stdout_fd, data)
except OSError:
# stdout closed, ignore
pass
# Forward stdin
if stdin_fd in rlist:
try:
data = os.read(stdin_fd, 1024)
os.write(master_fd, data)
except OSError:
# Either stdin EOF -> no data to pass
# or child died -> wait for master_fd to handle
pass
return b"".join(output_chunks)
def _get_passwd(user: str) -> pwd.struct_passwd:
try:
return pwd.getpwnam(user)
except KeyError as error:
raise errors.UserNotFoundError(user) from error
+90
View File
@@ -0,0 +1,90 @@
import shlex
class SourceError(Exception):
"""
Error raised manually from the user's source.
"""
class FSInstallationFailedError(Exception):
"""
Error raised when trying to install a file/directory to a target.
"""
def __init__(self, source: str, target: str, reason: str):
self.source = source
self.target = target
super().__init__(f"Failed to install file from {source} to {target}: {reason}.")
class FSSymlinkFailedError(Exception):
"""
Error raised when trying to create a symlink to a target.
"""
def __init__(self, link_name: str, target: str, reason: str):
self.link_name = link_name
self.target = target
super().__init__(f"Failed to install symlink from {link_name} to {target}: {reason}.")
class InvalidOnDisableError(Exception):
"""
Error raised when trying to create a Module with an invalid on_disable method.
"""
def __init__(self, module: str, reason: str):
self.module = module
self.reason = reason
super().__init__(
f"Module '{module}' contains an invalid on_disable method. Reason: {reason}."
)
class UserNotFoundError(Exception):
"""
Raised when a specified user cannot be found in the system.
Attributes:
user (str): The user that caused the exception.
"""
def __init__(self, user: str) -> None:
self.user = user
super().__init__(f"The user '{user}' doesn't exist.")
class GroupNotFoundError(Exception):
"""
Raised when a specified group cannot be found in the system.
Attributes:
group (str): The group that caused the exception.
"""
def __init__(self, group: str) -> None:
self.group = group
super().__init__(f"The group '{group}' doesn't exist.")
class CommandFailedError(Exception):
"""
Raised when running a command failed.
Attributes:
command (list[str]): The command that caused the exception.
exit_code (int): The exit code of the command
output (str|None): Output of the command.
"""
def __init__(self, command: list[str], exit_code: int, output: str | None) -> None:
self.command = shlex.join(command)
self.exit_code = exit_code
if output:
self.output: str | None = output.strip()
else:
self.output = None
super().__init__(
f"Command '{self.command}' returned with a non-zero exit code {self.exit_code}."
)
+221
View File
@@ -0,0 +1,221 @@
import os
import typing
import decman.core.error as errors
import decman.core.fs as fs
import decman.core.module as module
import decman.core.output as output
import decman.core.store as _store
def update_files(
store: _store.Store,
modules: list[module.Module],
files: dict[str, fs.File],
directories: dict[str, fs.Directory],
symlinks: dict[str, str | fs.Symlink],
dry_run: bool = False,
) -> bool:
"""
Apply the desired file and directory state.
Installs common and module-provided files and directories, tracks all checked paths, detects
changes, removes files no longer managed, and updates the store.
On failure, no removals are performed and the store is left unchanged.
Arguments:
store:
Persistent store used to track managed file paths.
modules:
Enabled modules providing additional files and directories.
files:
Common files to install (target path -> File).
directories:
Common directories to install (target path -> Directory).
dry_run:
If True, perform change detection only without modifying the filesystem.
Returns:
True if all operations completed successfully, False if installation failed.
"""
all_checked_files = []
all_changed_files = []
store.ensure("all_files", [])
output.print_summary("Updating files.")
try:
output.print_debug("Applying common files.")
checked, changed = _install_files(files, dry_run=dry_run)
all_checked_files += checked
all_changed_files += changed
output.print_debug("Applying common directories.")
checked, changed = _install_directories(directories, dry_run=dry_run)
all_checked_files += checked
all_changed_files += changed
output.print_debug("Applying common symlinks.")
checked, changed = _install_symlinks(symlinks, dry_run=dry_run)
all_checked_files += checked
all_changed_files += changed
for mod in modules:
module_changed_files = []
output.print_debug(f"Applying files in module '{mod.name}'.")
checked, changed = _install_files(
mod.files(),
variables=mod.file_variables(),
dry_run=dry_run,
)
all_checked_files += checked
module_changed_files += changed
output.print_debug(f"Applying directories in module '{mod.name}'.")
checked, changed = _install_directories(
mod.directories(),
variables=mod.file_variables(),
dry_run=dry_run,
)
all_checked_files += checked
module_changed_files += changed
output.print_debug(f"Applying symlinks in module '{mod.name}'.")
checked, changed = _install_symlinks(
mod.symlinks(),
dry_run=dry_run,
)
all_checked_files += checked
module_changed_files += changed
if len(module_changed_files) > 0:
output.print_debug(
f"Module '{mod.name}' set to changed due to modified "
f"files: '{"', '".join(module_changed_files)}'."
)
mod._changed = True
all_changed_files += module_changed_files
except errors.FSInstallationFailedError as error:
output.print_error(str(error))
output.print_traceback()
return False
except errors.FSSymlinkFailedError as error:
output.print_error(str(error))
output.print_traceback()
return False
to_remove = []
for file in store["all_files"]:
if file not in all_checked_files:
to_remove.append(file)
output.print_list("Updated files:", all_changed_files, elements_per_line=1)
if not dry_run:
for file in to_remove:
try:
os.remove(file)
except OSError as error:
output.print_warning(f"Failed to remove file: '{file}': {error.strerror}.")
store["all_files"] = all_checked_files
output.print_list("Removed files:", to_remove, elements_per_line=1)
return True
def _install_files(
files: dict[str, fs.File],
variables: typing.Optional[dict[str, str]] = None,
dry_run: bool = False,
) -> tuple[list[str], list[str]]:
checked_files = []
changed_files = []
for target_filename, file in files.items():
output.print_debug(f"Checking file {target_filename}.")
checked_files.append(target_filename)
try:
if file.copy_to(target_filename, variables=variables, dry_run=dry_run):
changed_files.append(target_filename)
except FileNotFoundError as error:
raise errors.FSInstallationFailedError(
file.source_file or "content", target_filename, "Source file doesn't exist."
) from error
except OSError as error:
raise errors.FSInstallationFailedError(
file.source_file or "content", target_filename, error.strerror or str(error)
) from error
except UnicodeEncodeError as error:
raise errors.FSInstallationFailedError(
file.source_file or "content", target_filename, "Unicode encoding failed."
) from error
except UnicodeDecodeError as error:
raise errors.FSInstallationFailedError(
file.source_file or "content", target_filename, "Unicode decoding failed."
) from error
return checked_files, changed_files
def _install_directories(
directories: dict[str, fs.Directory],
variables: typing.Optional[dict[str, str]] = None,
dry_run: bool = False,
) -> tuple[list[str], list[str]]:
checked_files = []
changed_files = []
for target_dirname, directory in directories.items():
output.print_debug(f"Checking directory {target_dirname}.")
try:
checked, changed = directory.copy_to(
target_dirname, variables=variables, dry_run=dry_run
)
except FileNotFoundError as error:
raise errors.FSInstallationFailedError(
directory.source_directory,
target_dirname,
"Source directory doesn't exist.",
) from error
except OSError as error:
raise errors.FSInstallationFailedError(
directory.source_directory, target_dirname, error.strerror or str(error)
) from error
except UnicodeEncodeError as error:
raise errors.FSInstallationFailedError(
directory.source_directory, target_dirname, "Unicode encoding failed."
) from error
except UnicodeDecodeError as error:
raise errors.FSInstallationFailedError(
directory.source_directory, target_dirname, "Unicode decoding failed."
) from error
checked_files += checked
changed_files += changed
return checked_files, changed_files
def _install_symlinks(
symlinks: dict[str, str | fs.Symlink], dry_run: bool = False
) -> tuple[list[str], list[str]]:
checked_files = []
changed_files = []
for link_name, target in symlinks.items():
output.print_debug(f"Checking symlink {link_name}.")
checked_files.append(link_name)
target_link: fs.Symlink = target if type(target) is fs.Symlink else fs.Symlink(target) # type: ignore
if target_link.link_to(link_name, dry_run):
changed_files.append(link_name)
return checked_files, changed_files
+445
View File
@@ -0,0 +1,445 @@
import grp
import os
import shutil
import typing
import decman.core.command as command
import decman.core.error as errors
import decman.core.output as output
def create_missing_dirs(dirct: str, uid: typing.Optional[int], gid: typing.Optional[int]):
if not os.path.isdir(dirct):
parent_dir = os.path.dirname(dirct)
if not os.path.isdir(parent_dir):
create_missing_dirs(parent_dir, uid, gid)
output.print_debug(f"Creating directory '{dirct}'.")
os.mkdir(dirct)
if uid is not None:
assert gid is not None, "If uid is set, then gid is set."
os.chown(dirct, uid, gid)
class File:
"""
Declarative file specification describing how a file should be materialized at a target path.
Exactly one of ``source_file`` or ``content`` must be provided.
The file can be created by copying an existing source file or by writing provided content. For
text files, optional variable substitution is applied at copy time. Binary files are copied or
written verbatim and never undergo substitution.
Ownership, permissions, and parent directories are enforced on creation. Missing parent
directories are created recursively and assigned the same ownership as the file when specified.
Parameters:
``source_file``:
Path to an existing file to copy from. Mutually exclusive with ``content``.
``content``:
In-memory file contents to write. Mutually exclusive with ``source_file``.
``bin_file``:
If ``True``, treat the file as binary. Disables variable substitution and writes bytes
verbatim.
``encoding``:
Text encoding used when reading or writing non-binary files.
``owner``:
System user name to own the file and created parent directories.
``group``:
System group name to own the file and created parent directories.
``permissions``:
File mode applied to the target file (e.g. ``0o644``).
Raises:
``ValueError``
If both ``source_file`` and ``content`` are ``None`` or if both are set.
``UserNotFoundError``
If ``owner`` does not exist on the system.
``GroupNotFoundError``
If ``group`` does not exist on the system.
Notes:
Variable substitution is a simple string replacement where each key in ``variables`` is
replaced by its corresponding value. No escaping or templating semantics are applied.
"""
def __init__(
self,
source_file: typing.Optional[str] = None,
content: typing.Optional[str] = None,
bin_file: bool = False,
encoding: str = "utf-8",
owner: typing.Optional[str] = None,
group: typing.Optional[str] = None,
permissions: int = 0o644,
):
if source_file is None and content is None:
raise ValueError("Both source_file and content cannot be None.")
if source_file is not None and content is not None:
raise ValueError("Both source_file and content cannot be set.")
self.source_file = source_file
self.content = content
self.permissions = permissions
self.bin_file = bin_file
self.encoding = encoding
self.uid = None
self.gid = None
if owner is not None:
self.uid, self.gid = command.get_user_info(owner)
if group is not None:
try:
self.gid = grp.getgrnam(group).gr_gid
except KeyError as error:
raise errors.GroupNotFoundError(group) from error
def copy_to(
self, target: str, variables: typing.Optional[dict[str, str]] = None, dry_run: bool = False
) -> bool:
"""
Copies the contents of this file to the target file if they differ.
Parameters:
target:
Path to the target file on disk.
variables:
Optional mapping of literal substrings to replace in the text content before
writing. Ignored for binary files and when ``bin_file`` is True.
Returns:
True if the file contents were/would be created or modified.
False if the existing file already contained the desired contents.
Raises:
OSError
If directory creation, file I/O, permission changes, or ownership changes fail
(e.g. permission denied, missing parent path components, I/O errors).
FileNotFoundError
If ``source_file`` is set and does not exist.
UnicodeDecodeError
If a text file cannot be decoded using ``encoding``.
UnicodeEncodeError
If text content cannot be encoded using ``encoding``.
"""
if variables is None:
variables = {}
target_directory = os.path.dirname(target)
if not dry_run:
create_missing_dirs(target_directory, self.uid, self.gid)
changed = self._write_content(target, variables, dry_run)
if changed:
output.print_debug(f"File '{target}' changed.")
if self.uid is not None and not dry_run:
assert self.gid is not None, "If uid is set, then gid is set."
os.chown(target, self.uid, self.gid)
if not dry_run:
os.chmod(target, self.permissions)
return changed
def _write_content(self, target: str, variables: dict[str, str], dry_run: bool):
# Case 1: copy from source file directly (binary or no substitutions)
if self.source_file is not None and (self.bin_file or len(variables) == 0):
if os.path.exists(target):
with open(self.source_file, "rb") as src, open(target, "rb") as dst:
if src.read() == dst.read():
return False
if not dry_run:
shutil.copy(self.source_file, target)
return True
# Case 2: binary content from memory
if self.bin_file and self.content is not None:
desired_bytes = self.content.encode(encoding=self.encoding)
if os.path.exists(target):
with open(target, "rb") as file:
if file.read() == desired_bytes:
return False
if not dry_run:
with open(target, "wb") as file:
file.write(desired_bytes)
return True
# From here on: text modes with possible substitutions
# Case 3: text content from source file with substitutions
if self.source_file is not None:
with open(self.source_file, "rt", encoding=self.encoding) as src:
content = src.read()
for var, value in variables.items():
content = content.replace(var, value)
if os.path.exists(target):
with open(target, "rt", encoding=self.encoding) as file:
if file.read() == content:
return False
if not dry_run:
with open(target, "wt", encoding=self.encoding) as file:
file.write(content)
return True
# Case 4: text content from in-memory string with substitutions
assert self.content is not None, "Content should be set since source_file was not set."
content = self.content
for var, value in variables.items():
content = content.replace(var, value)
if os.path.exists(target):
with open(target, "rt", encoding=self.encoding) as file:
if file.read() == content:
return False
if not dry_run:
with open(target, "wt", encoding=self.encoding) as file:
file.write(content)
return True
class Symlink:
"""
Declarative specification for linking a source to a destination.
Parameters:
``target``:
Path to an existing file to serve as the target of the symlink.
``owner``:
User name to own created parent directories.
``group``:
Group name to own created parent directories.
Raises:
``UserNotFoundError``
If ``owner`` does not exist on the system.
``GroupNotFoundError``
If ``group`` does not exist on the system.
"""
def __init__(
self,
target: str,
owner: typing.Optional[str] = None,
group: typing.Optional[str] = None,
):
self.target = target
self.owner = owner
self.group = group
self.uid = None
self.gid = None
if owner is not None:
self.uid, self.gid = command.get_user_info(owner)
if group is not None:
try:
self.gid = grp.getgrnam(group).gr_gid
except KeyError as error:
raise errors.GroupNotFoundError(group) from error
def link_to(self, link_name: str, dry_run: bool = False) -> bool:
"""
Creates a symlink ``link_name`` -> ``target``.
Parameters:
``link_name``:
Path to the target file on disk.
Returns:
True if a new link was/would be created or modified.
False if the existing link already contained the desired target.
Raises:
FSSymlinkFailedError
If creating the symlink failed due to directory creation, file I/O, permission
changes, or ownership changes fail (e.g. permission denied, missing parent path
components, I/O errors).
"""
def _is_symlink_to(path: str, target: str) -> bool:
if not os.path.islink(path):
return False
return os.readlink(path) == target
output.print_debug(f"Checking symlink {link_name}.")
try:
if _is_symlink_to(link_name, self.target):
return False
if dry_run:
return True
target_directory = os.path.dirname(link_name)
create_missing_dirs(target_directory, self.uid, self.gid)
if os.path.lexists(link_name):
os.unlink(link_name)
os.symlink(self.target, link_name)
return True
except OSError as error:
raise errors.FSSymlinkFailedError(
link_name, self.target, error.strerror or str(error)
) from error
class Directory:
"""
Declarative specification for copying the contents of a source directory into a target
directory.
Files are copied using the :class:`File` abstraction, inheriting its ownership,
permissions, encoding, and binary/text behavior. Text files can optionally undergo
variable substitution before being written.
Parameters:
``source_directory``:
Path to the directory whose contents will be mirrored into the target.
``bin_files``:
If ``True``, treat all files as binary; disables variable substitution and copies bytes
verbatim.
``encoding``:
Text encoding used when reading or writing non-binary files.
``owner``:
System user name to own created files and directories.
``group``:
System group name to own created files and directories.
``permissions``:
File mode applied to created or updated files (e.g. ``0o644``).
Raises:
``UserNotFoundError``
If ``owner`` does not exist on the system.
``GroupNotFoundError``
If ``group`` does not exist on the system.
"""
def __init__(
self,
source_directory: str,
bin_files: bool = False,
encoding: str = "utf-8",
owner: typing.Optional[str] = None,
group: typing.Optional[str] = None,
permissions: int = 0o644,
):
self.source_directory = source_directory
self.bin_files = bin_files
self.encoding = encoding
self.permissions = permissions
self.owner = owner
self.group = group
self.uid = None
self.gid = None
if owner is not None:
self.uid, self.gid = command.get_user_info(owner)
if group is not None:
try:
self.gid = grp.getgrnam(group).gr_gid
except KeyError as error:
raise errors.GroupNotFoundError(group) from error
def copy_to(
self,
target_directory: str,
variables: typing.Optional[dict[str, str]] = None,
dry_run: bool = False,
) -> tuple[list[str], list[str]]:
"""
Copies the files in this directory to the target directory. Only replaces files that differ.
Parameters:
target_directory:
Destination directory root. Relative layout from the source is preserved beneath
this path.
variables:
Optional mapping of literal substrings to replace in text files before writing.
Ignored for binary files.
dry_run:
If ``True``, perform a dry-run: no files are written, but the list of files that
*would* be processed is returned.
Returns:
tuple[list[str], list[str]]
The first list contains always every file in the source, the second list depends on
``dry_run``
When ``dry_run`` is ``False``, the second list contains paths of files that were
created or whose contents were modified.
When ``dry_run`` is ``True``, the second list contains paths of all files that would
be considered for creation or modification (no changes are actually performed).
Raises:
OSError
If directory traversal or file I/O fails (e.g. permission denied).
FileNotFoundError
If ``source_directory`` does not exist or becomes unavailable.
UnicodeDecodeError
If a text file cannot be decoded using ``encoding``.
UnicodeEncodeError
If text content cannot be encoded using ``encoding``.
"""
checked = []
changed_or_created = []
original_wd = os.getcwd()
try:
os.chdir(self.source_directory)
for src_dir, _, src_files in os.walk("."):
for src_file in src_files:
src_path = os.path.join(src_dir, src_file)
file = File(
source_file=src_path,
bin_file=self.bin_files,
encoding=self.encoding,
owner=self.owner,
group=self.group,
permissions=self.permissions,
)
target = os.path.normpath(os.path.join(target_directory, src_path))
checked.append(target)
if file.copy_to(target, variables, dry_run):
changed_or_created.append(target)
finally:
os.chdir(original_wd)
return checked, changed_or_created
+200
View File
@@ -0,0 +1,200 @@
import builtins
import dis
import inspect
import os
import textwrap
import types
import typing
import decman.core.error as errors
import decman.core.fs as fs
import decman.core.store as _store
class Module:
"""
Unit for organizing related files, packages and other configuration.
Inherit this class to create your own modules.
Parameters:
name:
The name of the module. It must be unique.
"""
def __init__(self, name: str) -> None:
self.name = name
self._changed = False
def __init_subclass__(cls, **kwargs):
super().__init_subclass__(**kwargs)
m = cls.__dict__.get("on_disable")
if m is None:
return
if not isinstance(m, staticmethod):
raise errors.InvalidOnDisableError(
f"{cls.__module__}.{cls.__name__}",
"on_disable must be declared as @staticmethod",
)
func = m.__func__
_validate_on_disable(f"{cls.__module__}.{cls.__name__}", func)
def before_update(self, store: _store.Store):
"""
Override this method to run python code before updating the system.
``store`` can be used to save persistent data between decman runs.
Handle errors within this function. If an error should abort running decman,
raise SourceError or CommandFailedError.
"""
def after_update(self, store: _store.Store):
"""
Override this method to run python code after updating the system.
``store`` can be used to save persistent data between decman runs.
Handle errors within this function. If an error should abort running decman,
raise SourceError or CommandFailedError.
"""
def on_enable(self, store: _store.Store):
"""
Override this method to run python code when this module gets enabled.
``store`` can be used to save persistent data between decman runs.
Handle errors within this function. If an error should abort running decman,
raise SourceError or CommandFailedError.
"""
def on_change(self, store: _store.Store):
"""
Override this method to run python code after the contents of this module have been
changed in the source.
``store`` can be used to save persistent data between decman runs.
Handle errors within this function. If an error should abort running decman,
raise SourceError or CommandFailedError.
"""
@staticmethod
def on_disable():
"""
Override this method to run python code when this module gets disabled.
This code will get copied *as is* to a temporary file. Do not use external variables or
imports. If you must use imports, define them inside this method.
"""
def files(self) -> dict[str, fs.File]:
"""
Override this method to return files that should be installed as a part of this module.
"""
return {}
def directories(self) -> dict[str, fs.Directory]:
"""
Override this method to return directories that should be installed as a part of this
module.
"""
return {}
def symlinks(self) -> dict[str, str | fs.Symlink]:
"""
Override this method to return symlinks that should be created as a part of this
module.
"""
return {}
def file_variables(self) -> dict[str, str]:
"""
Override this method to return variables that should replaced with a new value inside
this module's text files.
"""
return {}
def __hash__(self) -> int:
return hash(self.name)
def __eq__(self, other: object) -> bool:
return isinstance(other, self.__class__) and other.name == self.name
def write_on_disable_script(mod_obj: Module, out_dir: str) -> str | None:
"""
Writes a on_disable script for the given module. Returns the path to that script.
Raises:
OSError
If creating the script file fails.
"""
cls: typing.Type[Module] = type(mod_obj)
# Get the descriptor so we can unwrap staticmethod
desc = cls.__dict__.get("on_disable")
if desc is None:
return None
# unwrap staticmethod to get the real function
if isinstance(desc, staticmethod):
func = desc.__func__
else:
func = desc # already a function
src = inspect.getsource(func)
src = textwrap.dedent(src)
# Build a standalone script that defines the function and calls it
script = f"""#!/usr/bin/env python3
# generated from {cls.__module__}.{cls.__name__}.on_disable
{src}
if __name__ == "__main__":
{func.__name__}()
"""
script_file = fs.File(content=script, permissions=0o755)
script_path = os.path.join(out_dir, f"{mod_obj.name}_on_disable.py")
script_file.copy_to(script_path)
return script_path
def _iter_code_objects(code: types.CodeType):
yield code
for const in code.co_consts:
if isinstance(const, types.CodeType):
yield from _iter_code_objects(const)
def _validate_on_disable(module_type: str, func: types.FunctionType) -> None:
# No args
if inspect.signature(func).parameters:
raise errors.InvalidOnDisableError(module_type, "on_disable must take no parameters")
bad_names: set[str] = set()
for code in _iter_code_objects(func.__code__):
# No closures anywhere (outer or nested)
if code.co_freevars:
raise errors.InvalidOnDisableError(
module_type, "on_disable must not close over outer variables"
)
# No non-builtin globals / nonlocals anywhere
for ins in dis.get_instructions(code):
if ins.opname in ("LOAD_GLOBAL", "LOAD_DEREF"):
name = ins.argval
if not hasattr(builtins, name):
bad_names.add(name)
if bad_names:
raise errors.InvalidOnDisableError(
module_type,
f"on_disable uses nonlocal/global names: {', '.join(sorted(bad_names))}",
)
+258
View File
@@ -0,0 +1,258 @@
import os
import shutil
import sys
import traceback
import typing
import decman.config as config
# ─────────────────────────────
# Visible (non-ANSI) constants
# ─────────────────────────────
_TAG_TEXT = "[DECMAN]"
_SPACING = " "
_CONTINUATION_PREFIX_TEXT = f"{_TAG_TEXT}{_SPACING} "
INFO = 1
SUMMARY = 2
# ─────────────────────────────
# Color / formatting helpers
# ─────────────────────────────
def has_ansi_support() -> bool:
"""
Returns True if the running terminal supports ANSI colors or if colors should be enabled.
"""
if os.environ.get("NO_COLOR") is not None:
return False
if os.environ.get("FORCE_COLOR") is not None:
return True
if not sys.stdout.isatty():
return False
term = os.environ.get("TERM", "")
return term not in ("", "dumb")
def _apply_color(code: str, text: str) -> str:
if not config.color_output:
return text
return f"{code}{text}\033[m"
def _tag() -> str:
if not config.color_output:
return _TAG_TEXT
return "[\033[1;35mDECMAN\033[m]"
def _continuation_prefix() -> str:
return f"{_tag()}{_SPACING} "
def _red(text: str) -> str:
return _apply_color("\033[91m", text)
def _yellow(text: str) -> str:
return _apply_color("\033[93m", text)
def _cyan(text: str) -> str:
return _apply_color("\033[96m", text)
def _green(text: str) -> str:
return _apply_color("\033[92m", text)
def _gray(text: str) -> str:
return _apply_color("\033[90m", text)
# ─────────────────────────────
# Printing helpers
# ─────────────────────────────
def print_continuation(msg: str, level: int = SUMMARY):
"""
Prints a message without a prefix.
"""
if level == SUMMARY or config.debug_output or not config.quiet_output:
print(f"{_continuation_prefix()}{msg}")
def print_error(error_msg: str):
"""
Prints an error message to the user.
"""
print(f"{_tag()} {_red('ERROR')}: {error_msg}")
def print_traceback():
"""
Prints the traceback to debug output.
"""
for line in traceback.format_exc().splitlines():
print_debug(line)
def print_warning(msg: str):
"""
Prints a warning to the user.
"""
print(f"{_tag()} {_yellow('WARNING')}: {msg}")
def print_summary(msg: str):
"""
Prints a summary message to the user.
"""
print(f"{_tag()} {_cyan('SUMMARY')}: {msg}")
def print_info(msg: str):
"""
Prints a detailed message to the user if verbose output is not disabled.
"""
if config.debug_output or not config.quiet_output:
print(f"{_tag()} INFO: {msg}")
def print_debug(msg: str):
"""
Prints a detailed message to the user if debug messages are enabled.
"""
if config.debug_output:
print(f"{_tag()} {_gray('DEBUG')}: {msg}")
def print_command_output(command_output: str):
"""
Prints command output prefixed with a DECMAN tag.
"""
for line in command_output.strip().split("\n"):
print_continuation(line.strip())
# ─────────────────────────────
# List printing
# ─────────────────────────────
def print_list(
msg: str,
list_to_print: list[str],
elements_per_line: typing.Optional[int] = None,
max_line_width: typing.Optional[int] = None,
limit_to_term_size: bool = True,
level: int = SUMMARY,
):
"""
Prints a summary message to the user along with a list of elements.
If the list is empty, prints nothing.
"""
if len(list_to_print) == 0:
return
list_to_print = list_to_print.copy()
if level == SUMMARY:
print_summary(msg)
elif level == INFO:
print_info(msg)
print_continuation("", level=level)
if elements_per_line is None:
elements_per_line = len(list_to_print)
if max_line_width is None:
max_line_width = 2**32
if limit_to_term_size:
visible_prefix_len = len(_CONTINUATION_PREFIX_TEXT)
max_line_width = shutil.get_terminal_size().columns - visible_prefix_len
lines = [list_to_print.pop(0)]
index = 0
elements_in_current_line = 1
while list_to_print:
next_element = list_to_print.pop(0)
can_fit_elements = elements_in_current_line + 1 <= elements_per_line
can_fit_text = len(lines[index]) + len(next_element) <= max_line_width
if can_fit_text and can_fit_elements:
lines[index] += f" {next_element}"
elements_in_current_line += 1
else:
lines.append(next_element)
index += 1
elements_in_current_line = 1
for line in lines:
print_continuation(line, level=level)
print_continuation("", level=level)
# ─────────────────────────────
# Prompts
# ─────────────────────────────
def prompt_number(
msg: str,
min_num: int,
max_num: int,
default: typing.Optional[int] = None,
) -> int:
"""
Prompts the user for an integer.
"""
while True:
i = input(f"{_tag()} {_green('PROMPT')}: {msg}").strip()
if default is not None and i == "":
return default
try:
num = int(i)
if min_num <= num <= max_num:
return num
except ValueError:
pass
print_error("Invalid input.")
def prompt_confirm(msg: str, default: typing.Optional[bool] = None) -> bool:
"""
Prompts the user for confirmation.
"""
options_suffix = "(y/n)"
if default is not None:
options_suffix = "(Y/n)" if default else "(y/N)"
while True:
i = input(f"{_tag()} {_green('PROMPT')} {options_suffix}: {msg} ").strip()
if default is not None and i == "":
return default
if i.lower() in ("y", "ye", "yes"):
return True
if i.lower() in ("n", "no"):
return False
print_error("Invalid input.")
+87
View File
@@ -0,0 +1,87 @@
import json
import os
import pathlib
import tempfile
import typing
class Store:
"""
Key-value store for saving decman state.
"""
def __init__(self, path: str, dry_run: bool = False) -> None:
self._store: dict[str, typing.Any] = {}
self._path = pathlib.Path(path)
self._dry_run = dry_run
if self._path.exists():
with self._path.open("rt", encoding="utf-8") as file:
self._store = json.load(file, object_hook=_decode_sets)
def __getitem__(self, key: str) -> typing.Any:
return self._store[key]
def __setitem__(self, key: str, value: typing.Any) -> None:
self._store[key] = value
def get(self, key: str, default: typing.Any = None) -> typing.Any:
return self._store.get(key, default)
def ensure(self, key: str, default: typing.Any = None):
if key not in self._store:
self._store[key] = default
def __enter__(self) -> "Store":
return self
def __exit__(self, exc_type, exc, tb):
self.save()
return False
def save(self) -> None:
"""
Saves the store to the defined path.
"""
if self._dry_run:
return
os.makedirs(self._path.parent, exist_ok=True)
with tempfile.NamedTemporaryFile(
"wt",
encoding="utf-8",
dir=self._path.parent,
delete=False,
) as tmp:
json.dump(self._store, tmp, cls=_SetJSONEncoder, indent=2)
tmp.flush()
os.fsync(tmp.fileno())
os.replace(tmp.name, self._path)
def __repr__(self) -> str:
return repr(self._store)
class _SetJSONEncoder(json.JSONEncoder):
def default(self, obj: typing.Any) -> typing.Any:
if isinstance(obj, set):
# generic, works for any set value
return {"__type__": "set", "items": list(obj)}
return super().default(obj)
def _decode_sets(obj: typing.Any) -> typing.Any:
if isinstance(obj, dict) and obj.get("__type__") == "set" and "items" in obj:
# decode lists inside sets as tuples
norm = []
for item in obj["items"]:
if isinstance(item, list):
norm.append(tuple(item))
else:
norm.append(item)
return set(norm)
return obj
-12
View File
@@ -1,12 +0,0 @@
"""
Errors used by decman.
"""
class UserFacingError(Exception):
"""
Execution of an important step failed and the program shouldn't continue.
"""
def __init__(self, user_facing_msg: str):
self.user_facing_msg = user_facing_msg
View File
+317
View File
@@ -0,0 +1,317 @@
import os
import pwd
import re
import subprocess
from dataclasses import dataclass
from typing import Literal, Optional
import decman
import decman.core.module as module
import decman.core.output as output
import decman.core.store as _store
from decman.core.error import CommandFailedError
OwnerTrust = Literal["never", "marginal", "full", "ultimate"]
SourceKind = Literal["fingerprint", "uri", "file"]
_TRUST_MAP = {
"never": "1",
"marginal": "2",
"full": "3",
"ultimate": "4",
}
_FPR_RE = re.compile(r"^[0-9A-F]{40}$")
@dataclass(frozen=True)
class Key:
fingerprint: str
source_kind: SourceKind
source: str # keyserver / uri / filepath
trust: Optional[OwnerTrust] = None
def __post_init__(self) -> None:
fpr = self.fingerprint.replace(" ", "").upper()
if not _FPR_RE.fullmatch(fpr):
raise ValueError(f"invalid OpenPGP fingerprint: {fpr}")
object.__setattr__(self, "fingerprint", fpr)
class _GPGInterface:
def __init__(self, user: str, home: str):
self.user = user
self.home = home
def ensure_home(self) -> bool:
"""
Returns True on succees. Returns False if the user doesn't exist.
"""
def create_missing_dirs(dirct: str, uid: int, gid: int):
dirct = os.path.normpath(dirct)
if not os.path.isdir(dirct):
parent_dir = os.path.dirname(dirct)
if not os.path.isdir(parent_dir):
create_missing_dirs(parent_dir, uid, gid)
os.mkdir(dirct)
os.chown(dirct, uid, gid)
os.chmod(dirct, 0o700)
try:
u = pwd.getpwnam(self.user)
create_missing_dirs(self.home, u.pw_uid, u.pw_gid)
return True
except OSError as error:
raise decman.SourceError(
f"Failed to create GPG directory {self.home} for {self.user}."
) from error
except KeyError:
return False
def list_fingerprints(self) -> set[str]:
out = decman.prg(
["gpg", "--homedir", self.home, "--batch", "--no-tty", "--with-colons", "--list-keys"],
user=self.user,
pty=False,
)
fprs: set[str] = set()
for line in out.splitlines():
if line.startswith("fpr:"):
parts = line.split(":")
if len(parts) > 9 and parts[9]:
fprs.add(parts[9])
return fprs
def set_key_trust(self, keys: list[tuple[str, OwnerTrust]]):
if not keys:
return
lines = [f"{fpr}:{_TRUST_MAP[trust]}:" for fpr, trust in keys]
data = "\n".join(lines) + "\n"
cmd = [
"gpg",
"--homedir",
self.home,
"--batch",
"--yes",
"--no-tty",
"--import-ownertrust",
]
# use subprocess manually since decman exposed functions don't allow setting input
p = subprocess.run(
cmd,
input=data,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
user=self.user,
text=True,
check=False,
)
if p.returncode != 0:
raise CommandFailedError(cmd, p.returncode, p.stdout)
def delete_keys(self, fingerprints: list[str]):
decman.prg(
[
"gpg",
"--homedir",
self.home,
"--batch",
"--yes",
"--no-tty",
"--delete-keys",
]
+ fingerprints,
user=self.user,
pty=False,
)
def fetch_key(self, uri: str):
decman.prg(
[
"gpg",
"--homedir",
self.home,
"--batch",
"--yes",
"--no-tty",
"--fetch-key",
uri,
],
user=self.user,
pty=False,
)
def import_key(self, path: str):
decman.prg(
[
"gpg",
"--homedir",
self.home,
"--batch",
"--yes",
"--no-tty",
"--import",
path,
],
user=self.user,
pty=False,
)
def receive_key(self, fingerprint: str, keyserver: str):
decman.prg(
[
"gpg",
"--homedir",
self.home,
"--batch",
"--yes",
"--no-tty",
"--keyserver",
keyserver,
"--recv-keys",
fingerprint,
],
user=self.user,
pty=False,
)
class GPGReceiver(module.Module):
"""
Module for receiving OpenPGP keys.
This is basically built for importing AUR package keys.
If trying to add a key to an user that doesn't exist, this module silently skips user.
It's functionality is limited and I don't recommend using this with your main user account.
Instead create specific account for AUR package building and import keys to that account.
This module doesn't use the GPGME library and instead just calls gpg directly. It's simpler and
good enough for this usecase.
This module is a singleton, meaning that you should create only a one instance of this module
and pass that around.
"""
def __init__(self) -> None:
super().__init__("gpgreceiver")
self._keys: dict[tuple[str, str], list[Key]] = {}
def receive_key(
self,
user: str,
gpg_home: str,
fingerprint: str,
keyserver: str,
trust: OwnerTrust | None = None,
):
"""
Receives a key.
The key is imported as the given ``user`` into the specified ``gpg_home``.
If trust is specified, sets it.
"""
self._keys.setdefault((user, gpg_home), []).append(
Key(fingerprint, "fingerprint", keyserver, trust)
)
def fetch_key(
self, user: str, gpg_home: str, fingerprint: str, uri: str, trust: OwnerTrust | None = None
):
"""
Fetches a key from a URI.
The key is imported as the given ``user`` into the specified ``gpg_home``.
If trust is specified, sets it.
"""
self._keys.setdefault((user, gpg_home), []).append(Key(fingerprint, "uri", uri, trust))
def import_key(
self, user: str, gpg_home: str, fingerprint: str, file: str, trust: OwnerTrust | None = None
):
"""
Imports a key from file.
The key is imported as the given ``user`` into the specified ``gpg_home``.
If trust is specified, sets it.
"""
self._keys.setdefault((user, gpg_home), []).append(Key(fingerprint, "file", file, trust))
def _add_key(self, gpg: _GPGInterface, key: Key):
match key.source_kind:
case "fingerprint":
gpg.receive_key(key.fingerprint, key.source)
case "uri":
gpg.fetch_key(key.source)
case "file":
gpg.import_key(key.source)
def before_update(self, store: _store.Store):
store.ensure("gpgreceiver_userhome_keys", {})
known_users = {
(line.split(":", 1)[0], line.split(":", 1)[1])
for line in store["gpgreceiver_userhome_keys"]
}
for user, gpg_home in self._keys.keys() | known_users:
keys = self._keys.get((user, gpg_home), [])
gpg = _GPGInterface(user, gpg_home)
if not gpg.ensure_home():
output.print_warning(f"User {user} doesn't exist, so PGP keys cannot be modified.")
del store["gpgreceiver_userhome_keys"][f"{user}:{gpg_home}"]
continue
old_fprs = store["gpgreceiver_userhome_keys"].get(f"{user}:{gpg_home}", set())
fprs_before_import = gpg.list_fingerprints()
new_fprs = set()
managed_fprs = set()
key_trust_levels = []
for key in keys:
managed_fprs.add(key.fingerprint)
if key.trust:
key_trust_levels.append((key.fingerprint, key.trust))
if key.fingerprint not in fprs_before_import:
output.print_info(
f"Adding PGP key {key.fingerprint} to {user}:{gpg_home} "
f"from {key.source_kind} {key.source}."
)
self._add_key(gpg, key)
new_fprs.add(key.fingerprint)
fprs_after_import = gpg.list_fingerprints()
missing = new_fprs - fprs_after_import
if missing:
raise decman.SourceError(
f"Fingerprints for PGP not found after importing all keys: {' '.join(missing)}"
)
if key_trust_levels:
gpg.set_key_trust(key_trust_levels)
unaccounted_fprs = (fprs_after_import - fprs_before_import) - new_fprs
if unaccounted_fprs:
output.print_warning(
"While adding PGP keys these fingerprints were unaccounted for: "
f"{' '.join(unaccounted_fprs)}"
)
output.print_warning("The keys were added, but their ownertrust was not set.")
fprs_to_remove = list(old_fprs - managed_fprs)
if fprs_to_remove:
output.print_list(
f"Deleting PGP keys from {user}:{gpg_home}", fprs_to_remove, level=output.INFO
)
gpg.delete_keys(fprs_to_remove)
store["gpgreceiver_userhome_keys"][f"{user}:{gpg_home}"] = managed_fprs
+387
View File
@@ -0,0 +1,387 @@
import grp
import pwd
from dataclasses import dataclass
from typing import Optional
import decman.core.command as command
import decman.core.module as module
import decman.core.output as output
import decman.core.store as _store
@dataclass(frozen=True)
class Group:
"""
Represents a group managed by the ``UserManager`` module.
The ``system`` attribute only affects the creation of this group.
After the group has been created, changing the ``system`` attribute does nothing.
"""
groupname: str
gid: Optional[int] = None
system: bool = False
def __str__(self) -> str:
parts = []
if self.gid is not None:
parts.append(f"gid={self.gid}")
if self.system:
parts.append("system")
return f"{self.groupname}({', '.join(parts)})"
@dataclass(frozen=True)
class User:
"""
Represents a user managed by the ``UserManager`` module.
The ``system`` attribute only affects the creation of this user.
After the user has been created, changing the ``system`` attribute does nothing.
"""
username: str
uid: Optional[int] = None
group: Optional[str] = None
home: Optional[str] = None
shell: Optional[str] = None
groups: tuple[str, ...] = ()
system: bool = False
def __str__(self) -> str:
parts = []
if self.uid is not None:
parts.append(f"uid={self.uid}")
if self.group is not None:
parts.append(f"gid={self.group}")
if self.home is not None:
parts.append(f"home={self.home}")
if self.shell is not None:
parts.append(f"shell={self.shell}")
if self.groups:
parts.append(f"groups={','.join(self.groups)}")
if self.system:
parts.append("system")
return f"{self.username}({', '.join(parts)})"
class UserManager(module.Module):
"""
A module for managing users and groups. This module is additive, if you create a user or a group
manually, this module will not modify them, unless you explicitly add them to this module.
Users and groups are created, modified and deleted at ``before_update`` -stage.
Users are added to groups and subuids/subgids at ``after_update`` -stage.
Decman store keys used by this module are:
- ``usermanager_users``
- ``usermanager_groups``
- ``usermanager_user_additional_groups``
- ``usermanager_user_subuids``
- ``usermanager_user_subgids``
Most management done by this module is with the commands ``useradd``, ``groupadd`` and
``usermod``.
This module contains useful utilities for the most common user management cases,
but it is not complete.
If you need advanced user management features you should probably fork this module.
This module is a singleton, meaning that you should create only a one instance of this module
and pass that around.
"""
def __init__(self) -> None:
super().__init__("usermanager")
self.users: set[User] = set()
self.groups: set[Group] = set()
self._user_additional_groups: dict[str, set[str]] = {}
self._user_subuids: dict[str, set[tuple[int, int]]] = {}
self._user_subgids: dict[str, set[tuple[int, int]]] = {}
def add_user(self, user: User):
"""
Ensures that the user exists with the given attributes.
"""
self.users.add(user)
def add_group(self, group: Group):
"""
Ensures that the group exists with the given attributes.
"""
self.groups.add(group)
def add_user_to_group(self, user: str, group: str):
"""
Ensures that the user is a member of the given group.
Both ``user`` and ``group`` should exist.
"""
self._user_additional_groups.setdefault(user, set()).add(group)
def add_subuids(self, user: str, first: int, last: int):
"""
Adds the range ``first``-``last`` subordinate uids to the ``user``s account.
Note!
This module doesn't parse ``/etc/subuid`` or ``/etc/subgid``.
Instead, the added subuids and subgids are stored in the decman store.
Stored values are used to remove the added subuids and subgids from the user.
Manual modifications or clearing the decman store can cause unexpected issues.
"""
self._user_subuids.setdefault(user, set()).add((first, last))
def add_subgids(self, user: str, first: int, last: int):
"""
Adds the range ``first``-``last`` subordinate gids to the ``user``s account.
Note!
This module doesn't parse ``/etc/subuid`` or ``/etc/subgid``.
Instead, the added subuids and subgids are stored in the decman store.
Stored values are used to remove the added subuids and subgids from the user.
Manual modifications or clearing the decman store can cause unexpected issues.
"""
self._user_subgids.setdefault(user, set()).add((first, last))
def _check_user(self, user: User, user_groups_index: dict[str, set[str]]):
userdb_name = None
userdb_uid = None
try:
userdb_name = pwd.getpwnam(user.username)
except KeyError:
pass
try:
if user.uid is not None:
userdb_uid = pwd.getpwuid(user.uid)
except KeyError:
pass
# Prioritize uid match. If uid matches but name doesn't, rename the user.
userdb = userdb_uid or userdb_name
if not userdb:
self._add_user(user)
else:
self._ensure_user_matches(user, userdb, user_groups_index)
def _add_user(self, user: User):
cmd = ["useradd"]
if user.uid is not None:
cmd += ["--uid", str(user.uid)]
if user.group:
cmd += ["--gid", user.group]
if user.home:
cmd += ["--create-home", "--home-dir", user.home]
if user.shell:
cmd += ["--shell", user.shell]
if user.groups:
cmd += ["--groups", ",".join(list(user.groups))]
if user.system:
cmd.append("--system")
cmd.append(user.username)
output.print_info(f"Creating user {user}.")
command.prg(cmd, pty=False)
def _ensure_user_matches(
self, user: User, userdb: pwd.struct_passwd, user_groups_index: dict[str, set[str]]
):
cmd = ["usermod"]
if user.username != userdb.pw_name:
cmd += ["--login", user.username]
if user.uid is not None and user.uid != userdb.pw_uid:
cmd += ["--uid", str(user.uid)]
if user.group and user.group != grp.getgrgid(userdb.pw_gid).gr_name:
cmd += ["--gid", user.group]
if user.home and user.home != userdb.pw_dir:
cmd += ["--move-home", "--home", user.home]
if user.shell and user.shell != userdb.pw_shell:
cmd += ["--shell", user.shell]
# Use old name to support renames, post rename groups match
old_groups = user_groups_index.get(userdb.pw_name, set())
if user.groups is not None and set(user.groups) != old_groups:
if user.groups:
cmd += ["--groups", ",".join(list(user.groups))]
elif old_groups:
# Remove user from other groups
cmd += ["-r", "--groups", ",".join(old_groups)]
if len(cmd) > 1:
# Use old name to support renames
cmd.append(userdb.pw_name)
output.print_info(f"Modifying user {user}.")
command.prg(cmd, pty=False)
def _user_groups_index(self) -> dict[str, set[str]]:
result: dict[str, set[str]] = {}
for gr in grp.getgrall():
group = gr.gr_name
for user in gr.gr_mem:
result.setdefault(user, set()).add(group)
return result
def _check_group(self, group: Group):
groupdb_name = None
groupdb_gid = None
try:
groupdb_name = grp.getgrnam(group.groupname)
except KeyError:
pass
try:
if group.gid is not None:
groupdb_gid = grp.getgrgid(group.gid)
except KeyError:
pass
groupdb = groupdb_gid or groupdb_name
if not groupdb:
self._add_group(group)
else:
self._ensure_group_matches(group, groupdb)
def _add_group(self, group: Group):
cmd = ["groupadd"]
if group.gid is not None:
cmd += ["--gid", str(group.gid)]
if group.system:
cmd.append("--system")
cmd.append(group.groupname)
output.print_info(f"Creating group {group}.")
command.prg(cmd, pty=False)
def _ensure_group_matches(self, group: Group, groupdb: grp.struct_group):
cmd = ["groupmod"]
if group.groupname != groupdb.gr_name:
cmd += ["--new-name", group.groupname]
if group.gid is not None and group.gid != groupdb.gr_gid:
cmd += ["--gid", str(group.gid)]
if len(cmd) > 1:
# Use old name to support renames
cmd.append(groupdb.gr_name)
output.print_info(f"Modifying group {group}.")
command.prg(cmd, pty=False)
def _modify_user_groups_subids(self, user: str, store: _store.Store):
store.ensure("usermanager_user_additional_groups", {})
store.ensure("usermanager_user_subuids", {})
store.ensure("usermanager_user_subgids", {})
old_groups = store["usermanager_user_additional_groups"].get(user, set())
old_subuids = store["usermanager_user_subuids"].get(user, set())
old_subgids = store["usermanager_user_subgids"].get(user, set())
new_groups = self._user_additional_groups.get(user, set())
new_subuids = self._user_subuids.get(user, set())
new_subgids = self._user_subgids.get(user, set())
groups_to_remove = old_groups - new_groups
groups_to_add = new_groups - old_groups
subuids_to_remove = old_subuids - new_subuids
subuids_to_add = new_subuids - old_subuids
subgids_to_remove = old_subgids - new_subgids
subgids_to_add = new_subgids - old_subgids
output.print_list(
f"Removing {user} from groups:", list(groups_to_remove), level=output.INFO
)
output.print_list(f"Adding {user} to groups:", list(groups_to_add), level=output.INFO)
# It's not possible to remove and add groups at the same time, so remove groups first
if groups_to_remove:
command.prg(["usermod", "-r", "-G", ",".join(groups_to_remove), user], pty=False)
# Set these only if things change, no need to clutter the store otherwise
store["usermanager_user_additional_groups"][user] = new_groups
# Rest of the changes can be done with a single command
cmd = ["usermod"]
if groups_to_add:
cmd += ["-a", "-G", ",".join(groups_to_add)]
for first, last in subuids_to_remove:
output.print_info(f"Removing subuids {first}-{last} from {user}.")
cmd += ["--del-subuids", f"{first}-{last}"]
for first, last in subuids_to_add:
output.print_info(f"Adding subuids {first}-{last} to {user}.")
cmd += ["--add-subuids", f"{first}-{last}"]
for first, last in subgids_to_remove:
output.print_info(f"Removing subgids {first}-{last} from {user}.")
cmd += ["--del-subgids", f"{first}-{last}"]
for first, last in subgids_to_add:
output.print_info(f"Adding subgids {first}-{last} to {user}.")
cmd += ["--add-subgids", f"{first}-{last}"]
if len(cmd) > 1:
cmd.append(user)
command.prg(cmd, pty=False)
# Set these only if things change, no need to clutter the store otherwise
store["usermanager_user_additional_groups"][user] = new_groups
store["usermanager_user_subuids"][user] = new_subuids
store["usermanager_user_subgids"][user] = new_subgids
def _delete_users_and_groups(self, store: _store.Store):
store.ensure("usermanager_users", set())
store.ensure("usermanager_groups", set())
managed_users = set(map(lambda u: u.username, self.users))
managed_groups = set(map(lambda g: g.groupname, self.groups))
groups_to_remove = store["usermanager_groups"] - managed_groups
users_to_remove = store["usermanager_users"] - managed_users
for user in users_to_remove:
output.print_info(f"Deleting user {user}.")
command.prg(["userdel", user], pty=False)
store["usermanager_users"] = managed_users
for group in groups_to_remove:
output.print_info(f"Deleting group {group}.")
command.prg(["groupdel", group], pty=False)
store["usermanager_groups"] = managed_groups
def before_update(self, store: _store.Store):
for group in self.groups:
self._check_group(group)
user_groups_index = self._user_groups_index()
for user in self.users:
self._check_user(user, user_groups_index)
self._delete_users_and_groups(store)
def after_update(self, store: _store.Store):
# Iterate all entries to ensure removals take place
for user in pwd.getpwall():
self._modify_user_groups_subids(user.pw_name, store)
-907
View File
@@ -1,907 +0,0 @@
"""
Library module for decman.
"""
import pwd
import shutil
import subprocess
import json
import os
import typing
import time
import decman.config as conf
import decman.error as err
import decman
_DECMAN_MSG_TAG = "[\033[1;35mDECMAN\033[m]"
_RED_PREFIX = "\033[91m"
_YELLOW_PREFIX = "\033[93m"
_CYAN_PREFIX = "\033[96m"
_GREEN_PREFIX = "\033[92m"
_GRAY_PREFIX = "\033[90m"
_RESET_SUFFIX = "\033[m"
_SPACING = " "
_CONTINUATION_PREFIX = f"{_DECMAN_MSG_TAG}{_SPACING} "
INFO = 1
SUMMARY = 2
def print_continuation(msg: str, level: int = SUMMARY):
"""
Prints a message without a prefix.
"""
if level == SUMMARY or conf.debug_output or not conf.quiet_output:
print(f"{_CONTINUATION_PREFIX}{msg}")
def print_error(error_msg: str):
"""
Prints an error message to the user.
"""
print(f"{_DECMAN_MSG_TAG} {_RED_PREFIX}ERROR{_RESET_SUFFIX}: {error_msg}")
def print_warning(msg: str):
"""
Prints a warning to the user.
"""
print(f"{_DECMAN_MSG_TAG} {_YELLOW_PREFIX}WARNING{_RESET_SUFFIX}: {msg}")
def print_summary(msg: str):
"""
Prints a summary message to the user.
"""
print(f"{_DECMAN_MSG_TAG} {_CYAN_PREFIX}SUMMARY{_RESET_SUFFIX}: {msg}")
def print_list(msg: str,
l: list[str],
elements_per_line: typing.Optional[int] = None,
max_line_width: typing.Optional[int] = None,
limit_to_term_size: bool = True,
level: int = SUMMARY):
"""
Prints a summary message to the user along with a list of elements.
If the list is empty, prints nothing.
"""
if len(l) == 0:
return
l = l.copy()
if level == SUMMARY:
print_summary(msg)
elif level == INFO:
print_info(msg)
print_continuation("", level=level)
if elements_per_line is None:
elements_per_line = len(l)
if max_line_width is None:
max_line_width = 2**32 # Big enough to basically be unlimited
if limit_to_term_size:
max_line_width = shutil.get_terminal_size().columns - len(
_SPACING) - len(_CONTINUATION_PREFIX)
lines = [f"{l.pop(0)}"]
index = 0
elements_in_current_line = 1
while l:
next_element = l.pop(0)
can_fit_elements = elements_in_current_line + 1 <= elements_per_line
can_fit_text = len(lines[index]) + len(next_element) <= max_line_width
if can_fit_text and can_fit_elements:
lines[index] += f" {next_element}"
elements_in_current_line += 1
else:
lines.append(f"{next_element}")
index += 1
elements_in_current_line = 1
for line in lines:
print_continuation(line, level=level)
print_continuation("", level=level)
def print_info(msg: str):
"""
Prints a detailed message to the user if verbose output is not disabled.
"""
if conf.debug_output or not conf.quiet_output:
print(f"{_DECMAN_MSG_TAG} INFO: {msg}")
def print_debug(msg: str):
"""
Prints a detailed message to the user if debug messages are enabled.
"""
if conf.debug_output:
print(f"{_DECMAN_MSG_TAG} {_GRAY_PREFIX}DEBUG{_RESET_SUFFIX}: {msg}")
def prompt_number(msg: str,
min_num: int,
max_num: int,
default: typing.Optional[int] = None) -> int:
"""
Prompts the user for a integer.
"""
while True:
i = input(
f"{_DECMAN_MSG_TAG} {_GREEN_PREFIX}PROMPT{_RESET_SUFFIX}: {msg}"
).strip()
if default is not None and i == "":
return default
try:
num = int(i)
if min_num <= num <= max_num:
return num
except ValueError:
pass
print_error("Invalid input.")
def prompt_confirm(msg: str, default: typing.Optional[bool] = None) -> int:
"""
Prompts the user for confirmation.
"""
options_suffix = "(y/n)"
if default is not None:
if default:
options_suffix = "(Y/n)"
else:
options_suffix = "(y/N)"
while True:
i = input(
f"{_DECMAN_MSG_TAG} {_GREEN_PREFIX}PROMPT{_RESET_SUFFIX} {options_suffix}: {msg} "
).strip()
if default is not None and i == "":
return default
if i.lower() in ("y", "ye", "yes"):
return True
if i.lower() in ("n", "no"):
return False
print_error("Invalid input.")
_STORE_SAVE_DIR = "/var/lib/decman/"
_STORE_SAVE_FILENAME = "/var/lib/decman/store.json"
class Store:
"""
Stores information between decman invocations.
This information is used for example to prevent re-enabling a service.
"""
def __init__(self):
self.source_file: typing.Optional[str] = None
self.allow_running_source_without_prompt: bool = False
self.enabled_systemd_units: list[str] = []
self._enabled_user_systemd_units: list[str] = []
self.enabled_modules: dict[str, str] = {}
self.created_files: list[str] = []
self.pkgbuild_latest_reviewed_commits: dict[str, str] = {}
self._package_file_cache: dict[str, list[tuple[str, str, int]]] = {}
def add_enabled_user_systemd_unit(self, user: str, unit: str):
"""
Stores a user unit as enabled.
"""
self._enabled_user_systemd_units.append(f"{user}->{unit}")
def remove_enabled_user_systemd_unit(self, user: str, unit: str):
"""
Removes a user unit from stored units.
"""
try:
self._enabled_user_systemd_units.remove(f"{user}->{unit}")
except ValueError:
pass
def is_systemd_used_unit_enabled(self, user: str, unit: str) -> bool:
"""
Returns true if the given user unit is stored as enabled.
"""
return f"{user}->{unit}" in self._enabled_user_systemd_units
def get_enabled_user_systemd_units(self) -> list[tuple[str, str]]:
"""
Returns all enabled systemd units.
"""
result = []
for unit_str in self._enabled_user_systemd_units:
unit_l = unit_str.split("->")
user = unit_l[0]
unit = unit_l[1]
result.append((user, unit))
return result
def get_package(self, package: str) -> typing.Optional[tuple[str, str]]:
"""
Returns the latest version and path of a package stored in the built packages cache as a
tuple (version, path).
"""
entries = self._package_file_cache.get(package)
if entries is None:
return None
latest_version = None
latest_path = None
latest_timestamp = 0
for entry in entries:
version, path, timestamp = entry
if latest_timestamp < timestamp and os.path.exists(path):
latest_timestamp = timestamp
latest_version = version
latest_path = path
print_debug(f"Latest file for {package} is '{latest_path}'.")
if latest_path is None:
return None
assert latest_version is not None, "If latest_path is set, then latest_version is set."
return (latest_version, latest_path)
def add_package_to_cache(self, package: str, version: str,
path_to_built_pkg: str):
"""
Adds a built package to the package file cache. Tries to remove excess cached packages.
"""
new_entry = (version, path_to_built_pkg, int(time.time()))
entries = self._package_file_cache.get(package, [])
for _, already_cached_path, __ in entries:
if already_cached_path == path_to_built_pkg:
print_debug(
f"Trying to cache {package} version {version}, but the version is already cached: {already_cached_path}"
)
return
entries.append(new_entry)
self._package_file_cache[package] = entries
self._clean_pkg_cache(package)
def _clean_pkg_cache(self, package: str):
oldest_path = None
oldest_timestamp = None
index_of_oldest = None
entries = self._package_file_cache[package]
print_debug(f"Package cache has {len(entries)} entries.")
if len(entries) <= conf.number_of_packages_stored_in_cache:
print_debug("Old files will not be removed.")
return
for index, entry in enumerate(entries):
_, path, timestamp = entry
if oldest_timestamp is None or oldest_timestamp > timestamp:
oldest_timestamp = timestamp
oldest_path = path
index_of_oldest = index
print_debug(f"Oldest cached file for {package} is '{oldest_path}'.")
if oldest_path is None:
return
assert index_of_oldest is not None
entries.pop(index_of_oldest)
if os.path.exists(oldest_path):
print_debug(f"Removing '{oldest_path}' from the package cache.")
try:
os.remove(oldest_path)
except OSError as e:
print_error(f"{e}")
print_error(
f"Failed to remove file '{oldest_path}' from the package cache."
)
print_continuation("You'll have to remove the file manually.")
self._package_file_cache[package] = entries
def save(self):
"""
Writes the store to a file.
"""
path = os.path.join(_STORE_SAVE_DIR, _STORE_SAVE_FILENAME)
print_debug(f"Writing Store to '{path}'.")
d = {
"source_file": self.source_file,
"allow_running_source_without_prompt":
self.allow_running_source_without_prompt,
"enabled_systemd_units": self.enabled_systemd_units,
"enabled_user_systemd_units": self._enabled_user_systemd_units,
"enabled_modules": self.enabled_modules,
"created_files": self.created_files,
"package_file_cache": self._package_file_cache,
"pkgbuild_git_commits": self.pkgbuild_latest_reviewed_commits
}
try:
os.makedirs(_STORE_SAVE_DIR, exist_ok=True)
with open(path, "wt", encoding="utf-8") as file:
json.dump(d, file)
except OSError as e:
print_error(f"{e}")
raise err.UserFacingError("Failed to save decman store.") from e
@staticmethod
def restore() -> "Store":
"""
Reads a saved Store from a file if it exists.
"""
path = os.path.join(_STORE_SAVE_DIR, _STORE_SAVE_FILENAME)
print_debug(f"Reading Store from '{path}'.")
try:
store = Store()
if not os.path.exists(path):
return store
with open(path, "rt", encoding="utf-8") as file:
d = json.load(file)
store.source_file = d.get("source_file", None)
store.allow_running_source_without_prompt = d.get(
"allow_running_source_without_prompt", False)
store.enabled_systemd_units = d.get(
"enabled_systemd_units",
[],
)
store._enabled_user_systemd_units = d.get(
"enabled_user_systemd_units",
[],
)
store.enabled_modules = d.get("enabled_modules", {})
store.created_files = d.get("created_files", [])
store._package_file_cache = d.get("package_file_cache", {})
store.pkgbuild_latest_reviewed_commits = d.get(
"pkgbuild_git_commits",
{},
)
return store
except json.JSONDecodeError as e:
print_error(f"{e}")
raise err.UserFacingError(
"Failed to parse decman store json.") from e
except OSError as e:
print_error(f"{e}")
raise err.UserFacingError(
"Failed to read saved decman store.") from e
class Source:
"""
Configuration that describes a system.
"""
def __init__(
self,
pacman_packages: set[str],
aur_packages: set[str],
user_packages: set[decman.UserPackage],
ignored_packages: set[str],
systemd_units: set[str],
systemd_user_units: dict[str, set[str]],
files: dict[str, decman.File],
directories: dict[str, decman.Directory],
modules: set[decman.Module],
):
self.pacman_packages = pacman_packages
self.aur_packages = aur_packages
self.user_packages = user_packages
self.ignored_packages = ignored_packages
self.systemd_units = systemd_units
self.systemd_user_units = systemd_user_units
self.files = files
self.directories = directories
self.modules = modules
def run_on_enable(self, store: Store):
"""
Runs on_enable of every module that was now enabled.
"""
for module in self.modules:
if module.enabled and module.name not in store.enabled_modules:
module.on_enable()
def run_on_disable(self, store: Store):
"""
Runs on_disable of every module that was now disabled.
"""
for module in self.modules:
if not module.enabled and module.name in store.enabled_modules:
module.on_disable()
def run_after_update(self):
"""
Runs after_update of every enabled module.
"""
for module in self.modules:
if module.enabled:
module.after_update()
def run_after_version_change(self, store: Store):
"""
Runs after_version_change of every enabled module that has it's version changed.
"""
for module in self.modules:
if module.enabled and module.version != store.enabled_modules.get(
module.name, module.version):
module.after_version_change()
elif module.enabled and module.name not in store.enabled_modules:
module.after_version_change()
def create_all_files(self, only_print: bool) -> list[str]:
"""
Creates all files and returns them. The files created are based on the specified files,
directories and modules.
"""
created_files = []
def install_files(files: dict[str, decman.File],
variables: typing.Optional[dict[str, str]] = None):
for target, file in files.items():
created_files.append(target)
if only_print:
continue
try:
print_debug(f"Installing file to {target}.")
file.copy_to(target, variables)
except OSError as e:
print_error(f"{e}")
raise err.UserFacingError(
f"Failed to install file to {target}.") from e
def install_dirs(dirs: dict[str, decman.Directory],
variables: typing.Optional[dict[str, str]] = None):
for target, directory in dirs.items():
try:
print_debug(f"Installing directory to {target}.")
created_files.extend(
directory.copy_to(target, variables, only_print))
except OSError as e:
print_error(f"{e}")
raise err.UserFacingError(
f"Failed to install directory to {target}.") from e
install_files(self.files)
install_dirs(self.directories)
for module in self.modules:
if module.enabled:
install_files(module.files(), module.file_variables())
install_dirs(module.directories(), module.file_variables())
return created_files
def all_file_targets(self) -> list[str]:
"""
Returns all file targets combined.
"""
all_files = []
all_files.extend(self.files.keys())
for module in self.modules:
if module.enabled:
all_files.extend(module.files().keys())
return all_files
def all_directory_targets(self) -> list[str]:
"""
Returns all directory targets combined.
"""
all_dirs = []
all_dirs.extend(self.directories.keys())
for module in self.modules:
if module.enabled:
all_dirs.extend(module.directories().keys())
return all_dirs
def files_to_remove(self, store: Store,
created_files: list[str]) -> list[str]:
"""
Returns all files that should be removed.
"""
to_remove = []
for path in store.created_files:
if path not in created_files:
to_remove.append(path)
return to_remove
def units_to_enable(self, store: Store) -> list[str]:
"""
Returns all systemd units that should be enabled.
"""
result = []
for unit in self._all_units():
if unit not in store.enabled_systemd_units:
result.append(unit)
return result
def units_to_disable(self, store: Store) -> list[str]:
"""
Returns all systemd units that should be disabled.
"""
result = []
for unit in store.enabled_systemd_units:
if unit not in self._all_units():
result.append(unit)
return result
def user_units_to_enable(self, store: Store) -> dict[str, list[str]]:
"""
Returns all user systemd units that should be enabled.
"""
result = {}
for user, units in self._all_user_units().items():
for unit in units:
if not store.is_systemd_used_unit_enabled(user, unit):
entry = result.get(user, [])
entry.append(unit)
result[user] = entry
return result
def user_units_to_disable(self, store: Store) -> dict[str, list[str]]:
"""
Returns all user systemd units that should be disabled.
"""
result = {}
for user, unit in store.get_enabled_user_systemd_units():
if unit not in self._all_user_units().get(user, set()):
entry = result.get(user, [])
entry.append(unit)
result[user] = entry
return result
def packages_to_remove(
self, currently_installed_packages: list[str]) -> list[str]:
"""
Returns all packages that should be removed. This includes pacman, aur and user packages.
"""
result = []
for pkg in currently_installed_packages:
if pkg in self.ignored_packages:
continue
if pkg not in self._all_pkgs():
result.append(pkg)
return result
def pacman_packages_to_install(
self, currently_installed_packages: list[str]) -> list[str]:
"""
Returns all pacman packages that should be installed.
"""
result = []
for pkg in self._all_pacman_pkgs():
if pkg in self.ignored_packages:
continue
if pkg not in currently_installed_packages:
result.append(pkg)
return result
def foreign_packages_to_install(
self, currently_installed_packages: list[str]) -> list[str]:
"""
Returns all aur and user packages that should be installed.
"""
result = []
for pkg in self._all_foreign_pkgs():
if pkg in self.ignored_packages:
continue
if pkg not in currently_installed_packages:
result.append(pkg)
return result
def all_enabled_modules(self) -> list[tuple[str, str]]:
"""
Returns all enabled modules and their versions.
"""
result = []
for module in self.modules:
if module.enabled:
result.append((module.name, module.version))
return result
def all_user_pkgs(self) -> set[decman.UserPackage]:
"""
Returns all active UserPackages.
"""
result = set()
result.update(self.user_packages)
for module in self.modules:
if module.enabled:
result.update(module.user_packages())
return result
def _all_pacman_pkgs(self) -> set[str]:
result = set()
result.update(self.pacman_packages)
for module in self.modules:
if module.enabled:
result.update(module.pacman_packages())
return result
def _all_foreign_pkgs(self) -> set[str]:
result = set()
result.update(self.aur_packages)
result.update(map(lambda p: p.pkgname, self.user_packages))
for module in self.modules:
if module.enabled:
result.update(module.aur_packages())
result.update(map(lambda p: p.pkgname, module.user_packages()))
return result
def _all_pkgs(self) -> set[str]:
result = set()
result.update(self._all_pacman_pkgs())
result.update(self._all_foreign_pkgs())
return result
def _all_units(self) -> set[str]:
result = set()
result.update(self.systemd_units)
for module in self.modules:
if module.enabled:
result.update(module.systemd_units())
return result
def _all_user_units(self) -> dict[str, set[str]]:
result = {}
result.update(self.systemd_user_units)
for module in self.modules:
if module.enabled:
result.update(module.systemd_user_units())
return result
class Pacman:
"""
Interface for interacting with pacman.
"""
def __init__(self):
self._installable = {}
def get_installed(self) -> list[str]:
"""
Returns a list of installed packages.
"""
try:
packages = subprocess.run(
conf.commands.list_pkgs(),
check=True,
stdout=subprocess.PIPE,
).stdout.decode().strip().split('\n')
return packages
except subprocess.CalledProcessError as error:
raise err.UserFacingError(
f"Failed to get installed packages using '{error.cmd}'. Output: {error.stdout}."
) from error
def is_installable(self, dep: str) -> bool:
"""
Returns True if a dependency can be installed using pacman.
"""
if dep in self._installable:
return self._installable[dep]
result = subprocess.run(conf.commands.is_installable(dep),
check=False,
capture_output=True).returncode == 0
self._installable[dep] = result
return result
def get_versioned_foreign_packages(self) -> list[tuple[str, str]]:
"""
Returns a list of installed packages and their versions that aren't from pacman databases,
basically AUR packages.
"""
try:
output = subprocess.run(
conf.commands.list_foreign_pkgs_versioned(),
check=True,
stdout=subprocess.PIPE).stdout.decode().strip().split('\n')
except subprocess.CalledProcessError as error:
raise err.UserFacingError(
f"Failed to get foreign packages using '{error.cmd}'. Output: {error.stdout}."
) from error
try:
return [(line.split(" ")[0], line.split(" ")[1])
for line in output]
except IndexError as error:
raise err.UserFacingError(
f"Failed to parse foreign packages from pacman output. Output: {output}"
) from error
def install(self, packages: list[str]):
"""
Installs the given packages.
"""
if not packages:
return
try:
subprocess.run(conf.commands.install_pkgs(packages), check=True)
subprocess.run(conf.commands.set_as_explicitly_installed(packages),
check=True,
capture_output=conf.suppress_command_output)
except subprocess.CalledProcessError as error:
raise err.UserFacingError(
"Failed to install packages using pacman.") from error
def install_dependencies(self, deps: list[str]):
"""
Installs the given dependencies.
"""
if not deps:
return
try:
subprocess.run(conf.commands.install_deps(deps), check=True)
except subprocess.CalledProcessError as error:
raise err.UserFacingError(
"Failed to install packages as dependencies using pacman."
) from error
def install_files(self, files: list[str], as_explicit: list[str]):
"""
Installs the given files first as dependencies. Then the packages listed in as_explicit are
installed explicitly.
"""
if not files:
return
try:
subprocess.run(conf.commands.install_files(files), check=True)
if as_explicit:
subprocess.run(
conf.commands.set_as_explicitly_installed(as_explicit),
check=True,
capture_output=conf.suppress_command_output)
except subprocess.CalledProcessError as error:
if conf.suppress_command_output:
print_error("Output:")
print_continuation(error.output)
raise err.UserFacingError(
"Failed to install package files using pacman.") from error
def upgrade(self):
"""
Upgrades all packages.
"""
try:
subprocess.run(conf.commands.upgrade(), check=True)
except subprocess.CalledProcessError as error:
raise err.UserFacingError(
"Failed to upgrade packages using pacman.") from error
def remove(self, packages: list[str]):
"""
Removes the given packages.
"""
if not packages:
return
try:
subprocess.run(conf.commands.remove(packages), check=True)
except subprocess.CalledProcessError as error:
raise err.UserFacingError(
"Failed to remove packages using pacman.") from error
class Systemd:
"""
Interface for interacting with systemd.
"""
def __init__(self, state: Store):
self.state = state
def enable_units(self, units: list[str]):
"""
Enables the given units.
"""
if not units:
return
try:
subprocess.run(conf.commands.enable_units(units),
check=True,
capture_output=conf.suppress_command_output)
except subprocess.CalledProcessError as error:
raise err.UserFacingError(
f"Failed to enable systemd units: {units}") from error
self.state.enabled_systemd_units += units
def disable_units(self, units: list[str]):
"""
Disables the given units.
"""
if not units:
return
try:
subprocess.run(conf.commands.disable_units(units),
check=True,
capture_output=conf.suppress_command_output)
except subprocess.CalledProcessError as error:
raise err.UserFacingError(
f"Failed to disable systemd units: {units}") from error
for unit in units:
try:
self.state.enabled_systemd_units.remove(unit)
except ValueError:
pass
def enable_user_units(self, units: list[str], user: str):
"""
Enables the given units for the given user.
"""
if not units:
return
try:
subprocess.run(conf.commands.enable_user_units(units, user),
check=True,
capture_output=conf.suppress_command_output)
except subprocess.CalledProcessError as error:
raise err.UserFacingError(
f"Failed to enable systemd units: {units} for {user}."
) from error
for unit in units:
self.state.add_enabled_user_systemd_unit(user, unit)
def disable_user_units(self, units: list[str], user: str):
"""
Disables the given units for the given user.
"""
if not units:
return
try:
subprocess.run(conf.commands.disable_user_units(units, user),
check=True,
capture_output=conf.suppress_command_output)
except subprocess.CalledProcessError as error:
raise err.UserFacingError(
f"Failed to disable systemd units: {units} for {user}."
) from error
for unit in units:
self.state.remove_enabled_user_systemd_unit(user, unit)
File diff suppressed because it is too large Load Diff
+98
View File
@@ -0,0 +1,98 @@
import importlib.metadata as metadata
import typing
import decman.core.module as module
import decman.core.store as _store
class Plugin:
"""
A Plugin manages one part of a system.
NAME:
Canonical plugin name.
"""
NAME: str = ""
def available(self) -> bool:
"""
Checks if this plugin can be enabled.
For example, this could check if a required command is available.
Returns true if this plugin can be enabled.
"""
return True
def apply(
self, store: _store.Store, dry_run: bool = False, params: list[str] | None = None
) -> bool:
"""
Ensures that the state managed by this plugin is present.
Set ``dry_run`` to only print changes applying this plugin would cause.
This method must not raise exceptions. Instead it should return False to indicate a
failure. The method should handle it's exceptions and print them to the user.
Returns ``True`` when applying was successful, ``False`` when it failed.
"""
return True
def process_modules(self, store: _store.Store, modules: list[module.Module]):
"""
Processes a module.
"""
def run_method_with_attribute(mod: module.Module, attribute: str) -> typing.Any:
"""
Runs the first method with the given attribute in the module and returns its returned value.
Returns ``None`` if no such method is found.
Only the first found method with the attribute is ran.
"""
for name in dir(mod):
attr = getattr(mod, name)
if not callable(attr):
continue
func = getattr(attr, "__func__", attr)
if getattr(func, attribute, False):
return attr()
return None
def run_methods_with_attribute(mod: module.Module, attribute: str) -> list[typing.Any]:
"""
Runs all methods with the given attribute in the module and returns their returned values.
Returns an empty list if no such methods are found.
"""
values = []
for name in dir(mod):
attr = getattr(mod, name)
if not callable(attr):
continue
func = getattr(attr, "__func__", attr)
if getattr(func, attribute, False):
values.append(attr())
return values
def available_plugins() -> dict[str, Plugin]:
"""
Returns all available plugins.
"""
plugins = {}
eps = metadata.entry_points(group="decman.plugins")
for ep in eps:
cls = ep.load()
if not issubclass(cls, Plugin):
continue
instance = cls()
if instance.available():
plugins[cls.NAME] = instance
return plugins
View File
-6
View File
@@ -1,6 +0,0 @@
import os
import sys
_SRC_PATH = os.path.join(os.path.dirname(__file__), "../src/")
sys.path.append(_SRC_PATH)
-3
View File
@@ -1,3 +0,0 @@
Simple text file with a %variable%
twice: %another_variable%
-3
View File
@@ -1,3 +0,0 @@
#!/usr/bin/env bash
# This file should be executable.
echo "Hello, world!"
-3
View File
@@ -1,3 +0,0 @@
1
1
1
-3
View File
@@ -1,3 +0,0 @@
2
2
2
Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.9 KiB

-3
View File
@@ -1,3 +0,0 @@
s1
s1
s1
-3
View File
@@ -1,3 +0,0 @@
s2
s2
s2
-61
View File
@@ -1,61 +0,0 @@
import os
import shutil
import sys
# This test is manual. You'll have to verify the results manually.
# NOTE: Change this if you want to run this script.
user = "kk"
cd = os.path.dirname(os.path.abspath(__file__))
os.chdir(cd)
sys.path.append(os.path.join(cd, "../../src/."))
from decman import File, Directory
#if os.path.exists("/tmp/decman-files"):
# shutil.rmtree("/tmp/decman-files")
#os.makedirs("/tmp/decman-files")
f1 = File(source_file="src/f1.txt")
f1.copy_to(
"/tmp/decman-files/f1.txt",
variables={
"%variable%": "123",
"%another_variable%": "456"
},
)
f2 = File(source_file="src/f2.sh", permissions=0o744)
f2.copy_to("/tmp/decman-files/f2.sh", )
f3 = File(content="%variable% doesn't work here.", bin_file=True)
f3.copy_to(
"/tmp/decman-files/f3.txt",
variables={
"%variable%": "123",
},
)
f4 = File(content="%variable% works here.", bin_file=False, owner=user)
f4.copy_to(
"/tmp/decman-files/f4.txt",
variables={
"%variable%": "123",
},
)
f5 = File(content="%variable% works here.",
bin_file=False,
owner=user,
group="root")
f5.copy_to(
"/tmp/decman-files/f5.txt",
variables={
"%variable%": "123",
},
)
d = Directory("src/srcdir", bin_files=True)
d.copy_to("/tmp/decman-files/targetdir")
+351
View File
@@ -0,0 +1,351 @@
import argparse
import types
import pytest
import decman.app as app # adjust if run_decman lives elsewhere
class DummyStore:
def __init__(self, enabled=None, scripts=None):
self._data = {}
if enabled is not None:
self._data["enabled_modules"] = list(enabled)
if scripts is not None:
self._data["module_on_disable_scripts"] = dict(scripts)
def __getitem__(self, key):
return self._data[key]
def __setitem__(self, key, value):
self._data[key] = value
def ensure(self, key, default):
self._data.setdefault(key, default)
class DummyModule:
def __init__(self, name):
self.name = name
self._changed = False
self.before_update_called = False
self.on_enable_called = False
self.on_change_called = False
self.after_update_called = False
def before_update(self, store):
self.before_update_called = True
def on_enable(self, store):
self.on_enable_called = True
def on_change(self, store):
self.on_change_called = True
def after_update(self, store):
self.after_update_called = True
@staticmethod
def on_disable():
print("Disabled")
class DummyPlugin:
def __init__(self, apply_result=True):
self.process_modules_called = False
self.apply_called_with = None
self.apply_result = apply_result
def process_modules(self, store, modules):
self.process_modules_called = True
def apply(self, store, dry_run=False, params=None):
self.apply_called_with = dry_run
return self.apply_result
def make_args(
only=None,
skip=None,
dry_run=False,
no_hooks=False,
):
return argparse.Namespace(
only=only, skip=skip or [], dry_run=dry_run, no_hooks=no_hooks, params=[]
)
@pytest.fixture
def no_op_output(monkeypatch):
ns = types.SimpleNamespace(
print_debug=lambda *a, **k: None,
print_summary=lambda *a, **k: None,
print_info=lambda *a, **k: None,
print_warning=lambda *a, **k: None,
)
monkeypatch.setattr(app, "output", ns)
return ns
@pytest.fixture
def base_decman(monkeypatch):
# Ensure decman attribute exists on app and has the fields we need
dm = types.SimpleNamespace()
dm.execution_order = []
dm.modules = []
dm.files = []
dm.directories = []
dm.symlinks = {}
dm.plugins = {}
dm.prg_calls = []
def prg(cmd):
dm.prg_calls.append(cmd)
dm.prg = prg
monkeypatch.setattr(app, "decman", dm)
return dm
@pytest.fixture
def file_manager(monkeypatch):
fm = types.SimpleNamespace()
fm.update_files_calls = []
fm.result = True
def update_files(store, modules, files, directories, symlinks, dry_run=False):
fm.update_files_calls.append(
dict(
store=store,
modules=list(modules),
files=list(files),
directories=list(directories),
symlinks=list(symlinks),
dry_run=dry_run,
)
)
return fm.result
fm.update_files = update_files
monkeypatch.setattr(app, "file_manager", fm)
return fm
def test_execution_order_only_and_skip(no_op_output, base_decman, file_manager):
base_decman.execution_order = ["files", "plugin_a", "plugin_b"]
args = make_args(
only=["files", "plugin_b"],
skip=["plugin_b"],
dry_run=True,
no_hooks=True,
)
store = DummyStore()
plugin = DummyPlugin(apply_result=False)
base_decman.plugins = {"plugin_b": plugin}
result = app.run_decman(store, args)
assert result is True
# Should have run only "files"
assert len(file_manager.update_files_calls) == 1
assert file_manager.update_files_calls[0]["dry_run"] is True
def test_returns_false_when_update_files_fails_and_skips_plugins(
no_op_output, base_decman, file_manager
):
base_decman.execution_order = ["files", "plugin_a"]
plugin = DummyPlugin(apply_result=True)
base_decman.plugins = {"plugin_a": plugin}
file_manager.result = False # update_files fails
args = make_args(dry_run=False, no_hooks=True)
store = DummyStore()
result = app.run_decman(store, args)
assert result is False
# update_files called once
assert len(file_manager.update_files_calls) == 1
# plugin should never be touched
assert plugin.process_modules_called is False
assert plugin.apply_called_with is None
def test_plugin_failure_returns_false(no_op_output, base_decman, file_manager):
base_decman.execution_order = ["plugin_a"]
plugin = DummyPlugin(apply_result=False)
base_decman.plugins = {"plugin_a": plugin}
args = make_args(dry_run=False, no_hooks=True)
store = DummyStore()
result = app.run_decman(store, args)
assert result is False
assert plugin.process_modules_called is True
assert plugin.apply_called_with is False
# No file updates
assert file_manager.update_files_calls == []
def test_disabled_modules_run_on_disable_script(no_op_output, base_decman, file_manager):
# enabled_modules contains a module that no longer exists
store = DummyStore(
enabled=["present", "old_mod"],
scripts={"old_mod": "/tmp/on_disable.sh"},
)
# Only "present" exists now, so "old_mod" is disabled
base_decman.modules = [DummyModule("present")]
base_decman.execution_order = []
args = make_args(dry_run=False, no_hooks=False)
result = app.run_decman(store, args)
assert result is True
# prg should be called with the script for old_mod
assert base_decman.prg_calls == [["/tmp/on_disable.sh"]]
assert store["enabled_modules"] == ["present"]
assert store["module_on_disable_scripts"] == {}
def test_on_disable_not_run_in_dry_run(no_op_output, base_decman, file_manager):
store = DummyStore(
enabled=["present", "old_mod"],
scripts={"old_mod": "/tmp/on_disable.sh"},
)
base_decman.modules = [DummyModule("present")]
base_decman.execution_order = []
args = make_args(dry_run=True, no_hooks=True)
result = app.run_decman(store, args)
assert result is True
# dry_run: on_disable scripts must not be executed
assert base_decman.prg_calls == []
def test_hooks_called_for_new_and_changed_modules(
no_op_output, base_decman, file_manager, monkeypatch, tmp_path
):
m1 = DummyModule("mod1")
m2 = DummyModule("mod2")
m1._changed = True
m2._changed = False
base_decman.modules = [m1, m2]
base_decman.execution_order = [] # no steps, just hooks
monkeypatch.setattr("decman.config.module_on_disable_scripts_dir", tmp_path)
# Only mod2 was previously enabled, so mod1 is "new"
store = DummyStore(enabled=["mod2"])
args = make_args(dry_run=False, no_hooks=False)
result = app.run_decman(store, args)
assert result is True
# before_update for all modules
assert m1.before_update_called is True
assert m2.before_update_called is True
# on_enable only for new module (mod1)
assert m1.on_enable_called is True
assert m2.on_enable_called is False
# on_change only for modules with _changed
assert m1.on_change_called is True
assert m2.on_change_called is False
# after_update for all modules
assert m1.after_update_called is True
assert m2.after_update_called is True
assert store["enabled_modules"] == ["mod2", "mod1"]
assert store["module_on_disable_scripts"] == {"mod1": str(tmp_path / "mod1_on_disable.py")}
def test_hooks_not_called_when_no_hooks(no_op_output, base_decman, file_manager):
m1 = DummyModule("mod1")
m1._changed = True
base_decman.modules = [m1]
base_decman.execution_order = []
store = DummyStore(enabled=["mod1"])
args = make_args(dry_run=False, no_hooks=True)
result = app.run_decman(store, args)
assert result is True
assert m1.before_update_called is False
assert m1.on_enable_called is False
assert m1.on_change_called is False
assert m1.after_update_called is False
def test_dry_run_skips_all_hooks_but_runs_steps_with_flag(no_op_output, base_decman, file_manager):
m1 = DummyModule("mod1")
m1._changed = True
base_decman.modules = [m1]
base_decman.execution_order = ["files", "plugin_a"]
plugin = DummyPlugin(apply_result=True)
base_decman.plugins = {"plugin_a": plugin}
store = DummyStore()
args = make_args(dry_run=True, no_hooks=False)
result = app.run_decman(store, args)
assert result is True
# Steps executed with dry_run=True
assert len(file_manager.update_files_calls) == 1
assert file_manager.update_files_calls[0]["dry_run"] is True
assert plugin.process_modules_called is True
assert plugin.apply_called_with is True
# All hooks skipped due to dry_run
assert m1.before_update_called is False
assert m1.on_enable_called is False
assert m1.on_change_called is False
assert m1.after_update_called is False
def test_missing_plugin_emits_warning_but_continues(base_decman, file_manager, monkeypatch):
warnings = []
def warn(msg):
warnings.append(msg)
out = types.SimpleNamespace(
print_debug=lambda *a, **k: None,
print_summary=lambda *a, **k: None,
print_info=lambda *a, **k: None,
print_warning=warn,
)
monkeypatch.setattr(app, "output", out)
base_decman.execution_order = ["unknown_plugin"]
base_decman.plugins = {} # none available
store = DummyStore()
args = make_args(dry_run=True, no_hooks=True)
result = app.run_decman(store, args)
assert result is True
assert any("unknown_plugin" in w for w in warnings)
+144
View File
@@ -0,0 +1,144 @@
import json
import sys
import typing
import pytest
import decman.core.command as command
import decman.core.output
def test_prg_pty_true_uses_pty_run_and_check(monkeypatch: pytest.MonkeyPatch):
calls: dict[str, typing.Any] = {}
def fake_pty_run(cmd, user=None, env_overrides=None, pass_environment=None, mimic_login=False):
calls["pty_run"] = (cmd, user, env_overrides, mimic_login)
return 0, "ok"
def fake_check_run_result(cmd, result, include_output=None):
calls["check_run_result"] = (cmd, result)
return result
def fake_print_warning(msg: str):
raise AssertionError("print_warning must not be called when code == 0")
monkeypatch.setattr(command, "pty_run", fake_pty_run)
monkeypatch.setattr(command, "check_run_result", fake_check_run_result)
monkeypatch.setattr(decman.core.output, "print_warning", fake_print_warning)
out = decman.prg(
["echo", "hi"],
user="alice",
env_overrides={"FOO": "bar"},
mimic_login=True,
pty=True,
check=True,
)
assert out == "ok"
assert calls["pty_run"] == (["echo", "hi"], "alice", {"FOO": "bar"}, True)
assert calls["check_run_result"] == (["echo", "hi"], (0, "ok"))
def test_prg_pty_false_uses_run(monkeypatch: pytest.MonkeyPatch):
calls: dict[str, typing.Any] = {}
def fake_run(cmd, user=None, env_overrides=None, pass_environment=None, mimic_login=False):
calls["run"] = (cmd, user, env_overrides, mimic_login)
return 0, "no-pty"
def fake_check_run_result(cmd, result, include_output=None):
return result
def fake_print_warning(msg: str):
raise AssertionError("print_warning must not be called when code == 0")
monkeypatch.setattr(command, "run", fake_run)
monkeypatch.setattr(command, "check_run_result", fake_check_run_result)
monkeypatch.setattr(decman.core.output, "print_warning", fake_print_warning)
out = decman.prg(["true"], pty=False, check=True)
assert out == "no-pty"
assert calls["run"] == (["true"], None, None, False)
def test_prg_check_false_warns_on_nonzero(monkeypatch: pytest.MonkeyPatch):
calls: dict[str, typing.Any] = {}
def fake_run(cmd, user=None, env_overrides=None, pass_environment=None, mimic_login=False):
# non-zero exit code
return 3, "bad"
def fake_check_run_result(cmd, result, include_output=None):
raise AssertionError("check_run_result must not be called when check=False")
def fake_print_warning(msg: str):
calls["warning"] = msg
monkeypatch.setattr(command, "run", fake_run)
monkeypatch.setattr(command, "check_run_result", fake_check_run_result)
monkeypatch.setattr(decman.core.output, "print_warning", fake_print_warning)
out = decman.prg(["cmd", "arg"], pty=False, check=False)
assert out == "bad"
assert "cmd arg" in calls["warning"]
assert "exit code 3" in calls["warning"]
def test_prg_check_true_propagates_command_failed_error(monkeypatch: pytest.MonkeyPatch):
class CommandFailedError(Exception):
pass
def fake_run(cmd, user=None, env_overrides=None, pass_environment=None, mimic_login=False):
return 42, "boom"
def fake_check_run_result(cmd, result, include_output=None):
raise CommandFailedError((cmd, result))
def fake_print_warning(msg: str):
raise AssertionError("print_warning must not be called when check=True and error")
monkeypatch.setattr(command, "run", fake_run)
monkeypatch.setattr(command, "check_run_result", fake_check_run_result)
monkeypatch.setattr(decman.core.output, "print_warning", fake_print_warning)
with pytest.raises(CommandFailedError):
decman.prg(["boom"], pty=False, check=True)
def test_run_simple():
code, out = command.run([sys.executable, "-c", "print('ok')"])
assert code == 0
assert out.strip() == "ok"
def test_run_exec_failure():
code, out = command.run(["/does/not/exist"])
assert code != 0
assert "not" in out.lower()
def test_run_env_overrides_visible_in_child(monkeypatch):
code, out = command.run(
[
sys.executable,
"-c",
("import os, json; print(json.dumps({'FOO': os.environ['FOO'], }))"),
],
env_overrides={"FOO": "BAR"},
)
assert code == 0
data = json.loads(out.strip())
assert data["FOO"] == "BAR"
@pytest.mark.skipif(not sys.stdin.isatty(), reason="requires TTY")
def test_pty_run_simple():
code, out = command.pty_run([sys.executable, "-c", "print('ok')"])
assert code == 0
assert "ok" in out
assert "\r\n" not in out
+519
View File
@@ -0,0 +1,519 @@
import os
import pytest
import decman.core.error as errors
import decman.core.output as output
from decman.core.file_manager import (
_install_directories,
_install_files,
_install_symlinks,
update_files,
)
class DummyFile:
def __init__(self, result=True, exc: BaseException | None = None):
self.result = result
self.exc = exc
self.source_file = None
self.calls: list[tuple[str, dict | None, bool]] = []
def copy_to(self, target: str, variables=None, dry_run: bool = False) -> bool:
self.calls.append((target, variables, dry_run))
if self.exc is not None:
raise self.exc
return self.result
class DummyDirectory:
def __init__(
self,
checked: list[str] | None = None,
changed: list[str] | None = None,
exc: BaseException | None = None,
source_directory: str = "<src>",
):
self.checked = checked or []
self.changed = changed or []
self.exc = exc
self.source_directory = source_directory
self.calls: list[tuple[str, dict | None, bool]] = []
def copy_to(self, target: str, variables=None, dry_run: bool = False):
self.calls.append((target, variables, dry_run))
if self.exc is not None:
raise self.exc
return self.checked, self.changed
class DummyModule:
def __init__(
self,
name: str,
file_map: dict[str, DummyFile] | None = None,
dir_map: dict[str, DummyDirectory] | None = None,
symlink_map: dict[str, str] | None = None,
file_vars: dict[str, str] | None = None,
):
self.name = name
self._file_map = file_map or {}
self._dir_map = dir_map or {}
self._file_vars = file_vars or {}
self._symlink_map = symlink_map or {}
self._changed = False
def files(self):
return self._file_map
def directories(self):
return self._dir_map
def symlinks(self):
return self._symlink_map
def file_variables(self):
return self._file_vars
class DummyStore:
def __init__(self, initial: dict | None = None):
self._data = dict(initial or {})
def __getitem__(self, key):
return self._data[key]
def __setitem__(self, key, value):
self._data[key] = value
def ensure(self, key, default):
self._data.setdefault(key, default)
# ---- _install_files -------------------------------------------------------
def test_install_files_non_dry_run_tracks_checked_and_changed():
f1 = DummyFile(result=True)
f2 = DummyFile(result=False)
files = {
"/tmp/file1": f1,
"/tmp/file2": f2,
}
checked, changed = _install_files(files, variables={"X": "1"}, dry_run=False)
assert checked == ["/tmp/file1", "/tmp/file2"]
assert changed == ["/tmp/file1"]
assert f1.calls == [("/tmp/file1", {"X": "1"}, False)]
assert f2.calls == [("/tmp/file2", {"X": "1"}, False)]
def test_install_files_dry_run_uses_dry_run_flag_and_respects_return_value():
f1 = DummyFile(result=True)
f2 = DummyFile(result=False)
files = {
"/tmp/file1": f1,
"/tmp/file2": f2,
}
checked, changed = _install_files(files, variables=None, dry_run=True)
assert checked == ["/tmp/file1", "/tmp/file2"]
assert changed == ["/tmp/file1"] # only ones that "would" change
assert f1.calls == [("/tmp/file1", None, True)]
assert f2.calls == [("/tmp/file2", None, True)]
@pytest.mark.parametrize(
"exc",
[
FileNotFoundError("nope"),
OSError("boom"),
UnicodeEncodeError("utf-8", "x", 0, 1, "bad"),
UnicodeDecodeError("utf-8", b"x", 0, 1, "bad"),
],
)
def test_install_files_wraps_exceptions(exc):
f = DummyFile(exc=exc)
files = {"/tmp/file": f}
with pytest.raises(errors.FSInstallationFailedError) as e:
_install_files(files, dry_run=False)
msg = str(e.value)
assert "/tmp/file" in msg
assert "content" in msg or "Source file doesn't exist." in msg
# ---- _install_directories -------------------------------------------------
def test_install_directories_aggregates_checked_and_changed():
d1 = DummyDirectory(
checked=["/tmp/d1/a", "/tmp/d1/b"],
changed=["/tmp/d1/a"],
source_directory="/src/d1",
)
d2 = DummyDirectory(
checked=["/tmp/d2/a"],
changed=["/tmp/d2/a"],
source_directory="/src/d2",
)
dirs = {
"/tmp/d1": d1,
"/tmp/d2": d2,
}
checked, changed = _install_directories(dirs, variables={"Y": "2"}, dry_run=False)
assert checked == ["/tmp/d1/a", "/tmp/d1/b", "/tmp/d2/a"]
assert changed == ["/tmp/d1/a", "/tmp/d2/a"]
# dry_run flag and variables propagated
assert d1.calls == [("/tmp/d1", {"Y": "2"}, False)]
assert d2.calls == [("/tmp/d2", {"Y": "2"}, False)]
@pytest.mark.parametrize(
"exc",
[
FileNotFoundError("nope"),
OSError("boom"),
UnicodeEncodeError("utf-8", "x", 0, 1, "bad"),
UnicodeDecodeError("utf-8", b"x", 0, 1, "bad"),
],
)
def test_install_directories_wraps_exceptions(exc):
d = DummyDirectory(exc=exc, source_directory="/src")
dirs = {"/tmp/d": d}
with pytest.raises(errors.FSInstallationFailedError) as e:
_install_directories(dirs, dry_run=False)
msg = str(e.value)
assert "/tmp/d" in msg
assert "/src" in msg
# ---- update_files ---------------------------------------------------------
def test_update_files_success_updates_store_and_removes_stale_files(monkeypatch):
# Prepare common files/dirs
common_file = DummyFile(result=True)
common_dir = DummyDirectory(
checked=["/etc/app/config.d/a.conf"],
changed=["/etc/app/config.d/a.conf"],
source_directory="/src/config.d",
)
# Module with its own file
mod_file = DummyFile(result=True)
m = DummyModule(
name="mod1",
file_map={"/etc/app/mod1.conf": mod_file},
dir_map={},
file_vars={"FOO": "bar"},
)
# Store already has some files, including one stale file
store = DummyStore(
{"all_files": ["/etc/app/common.conf", "/etc/app/mod1.conf", "/etc/app/stale.conf"]}
)
removed = []
def fake_remove(path):
removed.append(path)
monkeypatch.setattr(os, "remove", fake_remove)
# Run
ok = update_files(
store=store,
modules={m},
files={"/etc/app/common.conf": common_file},
directories={"/etc/app/config.d": common_dir},
symlinks={},
dry_run=False,
)
assert ok is True
# common + dir content + module file were re-checked
assert set(store["all_files"]) == {
"/etc/app/common.conf",
"/etc/app/config.d/a.conf",
"/etc/app/mod1.conf",
}
# stale file should be removed
assert removed == ["/etc/app/stale.conf"]
# module marked changed because its file changed
assert m._changed is True
# copy_to called for all files with correct dry_run flag
assert common_file.calls == [("/etc/app/common.conf", None, False)]
assert mod_file.calls == [("/etc/app/mod1.conf", {"FOO": "bar"}, False)]
def test_update_files_dry_run_does_not_touch_store_or_remove(monkeypatch):
common_file = DummyFile(result=True)
common_dir = DummyDirectory(
checked=["/etc/app/config.d/a.conf"],
changed=["/etc/app/config.d/a.conf"],
source_directory="/src/config.d",
)
m = DummyModule(
name="mod1",
file_map={"/etc/app/mod1.conf": DummyFile(result=True)},
dir_map={},
)
store = DummyStore({"all_files": ["/etc/app/common.conf", "/etc/app/stale.conf"]})
removed = []
def fake_remove(path):
removed.append(path)
monkeypatch.setattr(os, "remove", fake_remove)
ok = update_files(
store=store,
modules={m},
files={"/etc/app/common.conf": common_file},
directories={"/etc/app/config.d": common_dir},
symlinks={},
dry_run=True,
)
assert ok is True
# Store unchanged
assert store["all_files"] == ["/etc/app/common.conf", "/etc/app/stale.conf"]
# No removals
assert removed == []
# copy_to called with dry_run=True
assert common_file.calls == [("/etc/app/common.conf", None, True)]
def test_update_files_propagates_fsinstallation_error_and_does_not_modify_store(monkeypatch):
# Use real store.Store to ensure interface compatibility if you prefer
store = DummyStore({"all_files": ["/etc/app/keep.conf"]})
# Fake failing _install_files
def failing_install_files(*args, **kwargs):
raise errors.FSInstallationFailedError("content", "/etc/app/broken.conf", "fail")
# Capture deletes
removed = []
def fake_remove(path):
removed.append(path)
# Spy on output error/traceback so they exist but don't blow up
error_msgs = []
def fake_print_error(msg):
error_msgs.append(msg)
traces = []
def fake_print_traceback():
traces.append(True)
import decman.core.file_manager as fm_mod
monkeypatch.setattr(fm_mod, "_install_files", failing_install_files)
monkeypatch.setattr(os, "remove", fake_remove)
monkeypatch.setattr(output, "print_error", fake_print_error)
monkeypatch.setattr(output, "print_traceback", fake_print_traceback)
ok = update_files(
store=store,
modules=set(),
files={"/etc/app/broken.conf": DummyFile()},
directories={},
symlinks={},
dry_run=False,
)
assert ok is False
# Store unchanged
assert store["all_files"] == ["/etc/app/keep.conf"]
# No deletions attempted
assert removed == []
# Error and traceback were logged
assert error_msgs
assert traces
# symlinks
def test_install_symlinks_creates_missing_link_and_parents(tmp_path):
target = tmp_path / "target"
target.write_text("x")
link = tmp_path / "a" / "b" / "link"
checked, changed = _install_symlinks({str(link): str(target)}, dry_run=False)
assert checked == [str(link)]
assert changed == [str(link)]
assert link.is_symlink()
assert os.readlink(link) == str(target)
def test_install_symlinks_no_change_when_already_points_to_target(tmp_path):
target = tmp_path / "target"
target.write_text("x")
link = tmp_path / "link"
os.symlink(str(target), str(link))
checked, changed = _install_symlinks({str(link): str(target)}, dry_run=False)
assert checked == [str(link)]
assert changed == []
assert link.is_symlink()
assert os.readlink(link) == str(target)
def test_install_symlinks_replaces_wrong_target(tmp_path):
target1 = tmp_path / "target1"
target2 = tmp_path / "target2"
target1.write_text("1")
target2.write_text("2")
link = tmp_path / "link"
os.symlink(str(target1), str(link))
checked, changed = _install_symlinks({str(link): str(target2)}, dry_run=False)
assert checked == [str(link)]
assert changed == [str(link)]
assert link.is_symlink()
assert os.readlink(link) == str(target2)
def test_install_symlinks_replaces_existing_regular_file(tmp_path):
target = tmp_path / "target"
target.write_text("x")
link = tmp_path / "link"
link.write_text("not a symlink")
checked, changed = _install_symlinks({str(link): str(target)}, dry_run=False)
assert checked == [str(link)]
assert changed == [str(link)]
assert link.is_symlink()
assert os.readlink(link) == str(target)
def test_install_symlinks_dry_run_does_not_touch_fs(tmp_path):
target = tmp_path / "target"
target.write_text("x")
link = tmp_path / "a" / "b" / "link"
checked, changed = _install_symlinks({str(link): str(target)}, dry_run=True)
assert checked == [str(link)]
assert changed == [str(link)] # would change
assert not link.exists()
def test_update_files_tracks_symlinks_and_removes_stale_symlinks(tmp_path):
# layout
root = tmp_path
t = root / "target"
t.write_text("x")
live_link = root / "links" / "live"
stale_link = root / "links" / "stale"
# pre-existing stale link to be removed
os.makedirs(stale_link.parent, exist_ok=True)
os.symlink(str(t), str(stale_link))
m = DummyModule(
name="mod1",
file_map={},
dir_map={},
symlink_map={str(live_link): str(t)},
)
store = DummyStore(
{"all_files": [str(stale_link)]} # new store key
)
ok = update_files(
store=store,
modules={m},
files={},
directories={},
symlinks={},
dry_run=False,
)
assert ok is True
# new link exists
assert live_link.is_symlink()
assert os.readlink(live_link) == str(t)
# stale link removed
assert not stale_link.exists()
# store updated
assert store["all_files"] == [str(live_link)]
def test_update_files_dry_run_does_not_create_or_remove_symlinks(tmp_path):
root = tmp_path
t = root / "target"
t.write_text("x")
live_link = root / "links" / "live"
stale_link = root / "links" / "stale"
os.makedirs(stale_link.parent, exist_ok=True)
os.symlink(str(t), str(stale_link))
m = DummyModule(
name="mod1",
file_map={},
dir_map={},
symlink_map={str(live_link): str(t)},
)
store = DummyStore({"all_files": [str(stale_link)]})
ok = update_files(
store=store,
modules={m},
files={},
directories={},
symlinks={},
dry_run=True,
)
assert ok is True
# no fs changes
assert not live_link.exists()
assert stale_link.is_symlink()
# store unchanged
assert store["all_files"] == [str(stale_link)]
+271
View File
@@ -0,0 +1,271 @@
import os
import stat
from pathlib import Path
# Adjust this import to match your actual module location
import decman.core.fs as fs
# --- fs.File tests --------------------------------------------------------------
def test_file_from_content_creates_and_is_idempotent(tmp_path: Path) -> None:
target = tmp_path / "file.txt"
f = fs.File(content="hello", permissions=0o600)
# First run: file must be created and reported as changed
changed1 = f.copy_to(str(target))
assert changed1 is True
assert target.read_text(encoding="utf-8") == "hello"
mode = stat.S_IMODE(target.stat().st_mode)
assert mode == 0o600
# Second run with same configuration: no content change
changed2 = f.copy_to(str(target))
assert changed2 is False
assert target.read_text(encoding="utf-8") == "hello"
assert stat.S_IMODE(target.stat().st_mode) == 0o600
def test_file_content_with_variables_and_change_detection(tmp_path: Path) -> None:
target = tmp_path / "templated.txt"
f = fs.File(content="hello {{NAME}}")
# First run: NAME=world
changed1 = f.copy_to(str(target), {"{{NAME}}": "world"})
assert changed1 is True
assert target.read_text(encoding="utf-8") == "hello world"
# Second run: same variables, no change
changed2 = f.copy_to(str(target), {"{{NAME}}": "world"})
assert changed2 is False
assert target.read_text(encoding="utf-8") == "hello world"
# Third run: different variables, should change
changed3 = f.copy_to(str(target), {"{{NAME}}": "there"})
assert changed3 is True
assert target.read_text(encoding="utf-8") == "hello there"
def test_file_from_source_text_with_and_without_variables(tmp_path: Path) -> None:
src = tmp_path / "src.txt"
src.write_text("VALUE={{X}}", encoding="utf-8")
target = tmp_path / "dst.txt"
# Without variables (raw copy)
f_raw = fs.File(source_file=str(src))
changed1 = f_raw.copy_to(str(target), {})
assert changed1 is True
assert target.read_text(encoding="utf-8") == "VALUE={{X}}"
# Idempotent raw copy
changed2 = f_raw.copy_to(str(target), {})
assert changed2 is False
# With variables (substitution)
f_sub = fs.File(source_file=str(src))
changed3 = f_sub.copy_to(str(target), {"{{X}}": "42"})
assert changed3 is True
assert target.read_text(encoding="utf-8") == "VALUE=42"
# Idempotent after substitution
changed4 = f_sub.copy_to(str(target), {"{{X}}": "42"})
assert changed4 is False
def test_file_binary_from_content(tmp_path: Path) -> None:
target = tmp_path / "bin.dat"
payload = b"\x00\x01\x02hello"
f = fs.File(content=payload.decode("latin1"), bin_file=True)
changed1 = f.copy_to(str(target))
assert changed1 is True
assert target.read_bytes() == payload
# Idempotent: second call does not rewrite
changed2 = f.copy_to(str(target))
assert changed2 is False
assert target.read_bytes() == payload
def test_file_binary_copy_from_source(tmp_path: Path) -> None:
src = tmp_path / "src.bin"
payload = b"\x10\x20\x30binary"
src.write_bytes(payload)
target = tmp_path / "dst.bin"
f = fs.File(source_file=str(src), bin_file=True)
changed1 = f.copy_to(str(target), {"IGNORED": "x"})
assert changed1 is True
assert target.read_bytes() == payload
# Idempotent, comparing bytes
changed2 = f.copy_to(str(target), {"IGNORED": "x"})
assert changed2 is False
assert target.read_bytes() == payload
def test_file_creates_parent_directories_and_applies_permissions(tmp_path: Path) -> None:
nested_dir = tmp_path / "a" / "b" / "c"
target = nested_dir / "file.txt"
f = fs.File(content="data", permissions=0o644)
changed = f.copy_to(str(target))
assert changed is True
assert target.read_text(encoding="utf-8") == "data"
# Directories created
assert nested_dir.is_dir()
# Permissions on file
mode = stat.S_IMODE(target.stat().st_mode)
assert mode == 0o644
# --- fs.Directory tests ---------------------------------------------------------
def _create_sample_source_tree(root: Path) -> None:
(root / "sub").mkdir(parents=True)
(root / "a.txt").write_text("A={{X}}", encoding="utf-8")
(root / "sub" / "b.txt").write_text("B={{X}}", encoding="utf-8")
def test_directory_copy_to_creates_and_is_idempotent(tmp_path: Path) -> None:
src_dir = tmp_path / "src"
dst_dir = tmp_path / "dst"
src_dir.mkdir()
_create_sample_source_tree(src_dir)
d = fs.Directory(
source_directory=str(src_dir),
bin_files=False,
encoding="utf-8",
permissions=0o644,
)
# First run: both fs should be created and reported as changed
checked1, changed1 = d.copy_to(str(dst_dir), variables={"{{X}}": "1"})
expected_paths = {
str(dst_dir / "a.txt"),
str(dst_dir / "sub" / "b.txt"),
}
assert set(changed1) == expected_paths
assert set(checked1) == expected_paths
assert (dst_dir / "a.txt").read_text(encoding="utf-8") == "A=1"
assert (dst_dir / "sub" / "b.txt").read_text(encoding="utf-8") == "B=1"
# Second run with same variables: no fs should be reported as changed
checked2, changed2 = d.copy_to(str(dst_dir), variables={"{{X}}": "1"})
assert changed2 == []
assert set(checked2) == expected_paths
def test_directory_copy_to_detects_changes_via_variables(tmp_path: Path) -> None:
src_dir = tmp_path / "src"
dst_dir = tmp_path / "dst"
src_dir.mkdir()
_create_sample_source_tree(src_dir)
d = fs.Directory(source_directory=str(src_dir))
# Initial materialization
_checked, changed1 = d.copy_to(str(dst_dir), variables={"{{X}}": "alpha"})
assert set(changed1) == {
str(dst_dir / "a.txt"),
str(dst_dir / "sub" / "b.txt"),
}
# Change variables -> both fs change
_checked, changed2 = d.copy_to(str(dst_dir), variables={"{{X}}": "beta"})
assert set(changed2) == {
str(dst_dir / "a.txt"),
str(dst_dir / "sub" / "b.txt"),
}
assert (dst_dir / "a.txt").read_text(encoding="utf-8") == "A=beta"
assert (dst_dir / "sub" / "b.txt").read_text(encoding="utf-8") == "B=beta"
def test_directory_copy_to_dry_run(tmp_path: Path) -> None:
src_dir = tmp_path / "src"
dst_dir = tmp_path / "dst"
src_dir.mkdir()
_create_sample_source_tree(src_dir)
d = fs.Directory(source_directory=str(src_dir))
# First, actually materialize once
d.copy_to(str(dst_dir), variables={"{{X}}": "1"})
# Now perform dry-run with different variables; contents must not change
before_a = (dst_dir / "a.txt").read_text(encoding="utf-8")
before_b = (dst_dir / "sub" / "b.txt").read_text(encoding="utf-8")
_checked, changed_dry = d.copy_to(
str(dst_dir),
variables={"{{X}}": "2"},
dry_run=True,
)
expected_paths = {
str(dst_dir / "a.txt"),
str(dst_dir / "sub" / "b.txt"),
}
assert set(changed_dry) == expected_paths
# Contents remain as before (no writes in dry-run)
assert (dst_dir / "a.txt").read_text(encoding="utf-8") == before_a
assert (dst_dir / "sub" / "b.txt").read_text(encoding="utf-8") == before_b
def test_directory_copy_to_restores_working_directory(tmp_path: Path) -> None:
src_dir = tmp_path / "src"
dst_dir = tmp_path / "dst"
src_dir.mkdir()
_create_sample_source_tree(src_dir)
d = fs.Directory(source_directory=str(src_dir))
original_cwd = os.getcwd()
try:
_checked, changed = d.copy_to(str(dst_dir), variables={"{{X}}": "x"})
assert set(changed) == {
str(dst_dir / "a.txt"),
str(dst_dir / "sub" / "b.txt"),
}
finally:
# Ensure the implementation restored CWD
assert os.getcwd() == original_cwd
def test_file_copy_to_dry_run(tmp_path):
target = tmp_path / "file.txt"
f = fs.File(content="hello", permissions=0o600)
# 1) Dry-run on non-existent file: would create -> returns True, no file written
assert not target.exists()
changed = f.copy_to(str(target), dry_run=True)
assert changed is True
assert not target.exists()
# 2) Actually create the file
changed_real = f.copy_to(str(target), dry_run=False)
assert changed_real is True
assert target.exists()
assert target.read_text(encoding="utf-8") == "hello"
# 3) Dry-run with same desired content: would NOT modify -> returns False, file unchanged
mtime_before = target.stat().st_mtime
changed_again = f.copy_to(str(target), dry_run=True)
assert changed_again is False
assert target.read_text(encoding="utf-8") == "hello"
# mtime must not change in dry-run
assert target.stat().st_mtime == mtime_before
+180
View File
@@ -0,0 +1,180 @@
import stat
import subprocess
import sys
from pathlib import Path
import pytest
import decman.core.error as errors
import decman.core.module as module
def test_module_without_on_disable_is_accepted():
class NoOnDisable(module.Module):
def __init__(self):
super().__init__("no_on_disable")
m = NoOnDisable()
assert m.name == "no_on_disable"
def test_on_disable_must_be_staticmethod():
with pytest.raises(errors.InvalidOnDisableError) as exc:
class NotStatic(module.Module):
def on_disable(): # type: ignore[no-redefined-builtin]
pass
msg = str(exc.value)
assert "on_disable must be declared as @staticmethod" in msg
def test_on_disable_must_take_no_parameters():
with pytest.raises(errors.InvalidOnDisableError) as exc:
class HasArgs(module.Module):
@staticmethod
def on_disable(x): # type: ignore[unused-argument]
pass
msg = str(exc.value)
assert "on_disable must take no parameters" in msg
SOME_CONST = 42 # noqa: F841
def test_on_disable_must_not_use_module_level_globals():
with pytest.raises(errors.InvalidOnDisableError) as exc:
class UsesGlobal(module.Module):
@staticmethod
def on_disable():
# will compile as LOAD_GLOBAL for SOME_CONST
print(SOME_CONST)
msg = str(exc.value)
assert "on_disable uses nonlocal/global names" in msg
assert "SOME_CONST" in msg
def test_on_disable_must_not_close_over_outer_variables():
# closure over outer local -> should be rejected via co_freevars on inner code
with pytest.raises(errors.InvalidOnDisableError) as exc:
class Closure(module.Module):
@staticmethod
def on_disable():
x = 1
def inner():
# closes over x
print(x) # pragma: no cover
inner()
msg = str(exc.value)
assert "must not close over outer variables" in msg
def test_on_disable_nested_function_without_closure_is_allowed():
class NestedNoClosure(module.Module):
def __init__(self):
super().__init__("nested_no_closure")
@staticmethod
def on_disable():
# nested function that only uses arguments / builtins
def inner(msg: str) -> None:
print(msg)
inner("OK")
# If the class definition above passed without raising, validation succeeded.
m = NestedNoClosure()
assert m.name == "nested_no_closure"
def test_on_disable_can_use_builtins_and_imports_inside_function():
class Valid(module.Module):
def __init__(self):
super().__init__("valid")
@staticmethod
def on_disable():
import math
print("sqrt2", round(math.sqrt(2), 3))
v = Valid()
assert v.name == "valid"
def test_write_on_disable_script_returns_none_when_no_on_disable(tmp_path):
class NoOnDisable(module.Module):
def __init__(self):
super().__init__("no_on_disable")
m = NoOnDisable()
script_path = module.write_on_disable_script(m, str(tmp_path))
assert script_path is None
assert not list(tmp_path.iterdir())
def test_write_on_disable_script_creates_executable_script(tmp_path):
class Simple(module.Module):
def __init__(self):
super().__init__("Simple")
@staticmethod
def on_disable():
print("ON_DISABLE_RUN")
m = Simple()
out_dir = tmp_path / "scripts"
out_dir.mkdir()
script_path_str = module.write_on_disable_script(m, str(out_dir))
assert script_path_str is not None
script_path = Path(script_path_str)
assert script_path.exists()
mode = script_path.stat().st_mode
assert mode & stat.S_IXUSR, "script must be executable by owner"
content = script_path.read_text(encoding="utf-8")
assert "generated from" in content
assert "def on_disable" in content
assert 'if __name__ == "__main__":' in content
# Execute the generated script and check its output
proc = subprocess.run(
[sys.executable, str(script_path)],
check=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
)
assert "ON_DISABLE_RUN" in proc.stdout
def test_write_on_disable_script_uses_module_and_class_in_header(tmp_path):
class HeaderCheck(module.Module):
def __init__(self):
super().__init__("HeaderCheck")
@staticmethod
def on_disable():
print("HEADER_CHECK")
m = HeaderCheck()
script_path_str = module.write_on_disable_script(m, str(tmp_path))
assert script_path_str is not None
script_path = Path(script_path_str)
content = script_path.read_text(encoding="utf-8")
# header should reference original module and class
assert f"{HeaderCheck.__module__}.{HeaderCheck.__name__}.on_disable" in content
+191
View File
@@ -0,0 +1,191 @@
import builtins
import types
import pytest
import decman.config as config
import decman.core.output as output
@pytest.fixture(autouse=True)
def reset_config():
# snapshot & restore config flags between tests
orig = types.SimpleNamespace(
debug_output=getattr(config, "debug_output", False),
quiet_output=getattr(config, "quiet_output", False),
color_output=getattr(config, "color_output", True),
)
yield
config.debug_output = orig.debug_output
config.quiet_output = orig.quiet_output
config.color_output = orig.color_output
def test_print_error_with_color_enabled(capsys):
config.color_output = True
output.print_error("boom")
out = capsys.readouterr().out
assert "boom" in out
assert "ERROR" in out
# crude check that some ANSI escapes are present
assert "\x1b[" in out
def test_print_error_with_color_disabled(capsys):
config.color_output = False
output.print_error("boom")
out = capsys.readouterr().out
assert out.strip().endswith("ERROR: boom")
# no ANSI escapes
assert "\x1b[" not in out
def test_print_info_respects_quiet_and_debug(capsys):
config.quiet_output = True
config.debug_output = False
output.print_info("msg 1")
out = capsys.readouterr().out
assert out == "" # suppressed
config.debug_output = True
output.print_info("msg 2")
out = capsys.readouterr().out
assert "INFO: msg 2" in out
config.quiet_output = False
config.debug_output = False
output.print_info("msg 3")
out = capsys.readouterr().out
assert "INFO: msg 3" in out
def test_print_debug_only_with_debug_enabled(capsys):
config.debug_output = False
output.print_debug("dbg")
assert capsys.readouterr().out == ""
config.debug_output = True
output.print_debug("dbg")
out = capsys.readouterr().out
assert "DEBUG" in out
assert "dbg" in out
def test_print_continuation_respects_level_and_config(capsys):
config.quiet_output = True
config.debug_output = False
output.print_continuation("x", level=output.INFO)
assert capsys.readouterr().out == ""
output.print_continuation("y", level=output.SUMMARY)
out = capsys.readouterr().out
assert "y" in out
def test_print_list_empty_outputs_nothing(capsys):
output.print_list("Header", [])
assert capsys.readouterr().out == ""
def test_print_list_summary_and_elements(capsys, monkeypatch):
# fixed terminal size for deterministic wrapping
monkeypatch.setattr(
output.shutil, "get_terminal_size", lambda: types.SimpleNamespace(columns=80)
)
config.quiet_output = False
config.debug_output = False
output.print_list("Installed packages:", ["a", "b", "c"])
out = capsys.readouterr().out.splitlines()
# header summary
assert any("SUMMARY" in line and "Installed packages:" in line for line in out)
# list content printed as continuation lines
assert any("a" in line for line in out)
assert any("b" in line for line in out)
assert any("c" in line for line in out)
def test_print_list_respects_elements_per_line_and_width(capsys, monkeypatch):
# very small width to force wrapping
monkeypatch.setattr(
output.shutil, "get_terminal_size", lambda: types.SimpleNamespace(columns=30)
)
items = [f"pkg{i}" for i in range(5)]
output.print_list(
"Pkgs:",
items,
elements_per_line=2,
limit_to_term_size=True,
level=output.SUMMARY,
)
out_lines = capsys.readouterr().out.splitlines()
list_lines = [l for l in out_lines if "pkg" in l]
# at most 2 per line
for line in list_lines:
assert len([p for p in items if p in line]) <= 2
def test_prompt_number_valid_input(monkeypatch):
inputs = iter(["3"])
monkeypatch.setattr(builtins, "input", lambda _: next(inputs))
res = output.prompt_number("Pick", 1, 5)
assert res == 3
def test_prompt_number_invalid_then_valid(monkeypatch, capsys):
inputs = iter(["foo", "10", "2"])
monkeypatch.setattr(builtins, "input", lambda _: next(inputs))
res = output.prompt_number("Pick", 1, 5)
assert res == 2
out = capsys.readouterr().out
# at least one error printed
assert "Invalid input" in out
def test_prompt_number_default_on_empty(monkeypatch):
inputs = iter([""])
monkeypatch.setattr(builtins, "input", lambda _: next(inputs))
res = output.prompt_number("Pick", 1, 5, default=4)
assert res == 4
@pytest.mark.parametrize(
"user_input,default,expected",
[
("y", None, True),
("Y", None, True),
("yes", None, True),
("n", None, False),
("No", None, False),
("", True, True),
("", False, False),
],
)
def test_prompt_confirm(monkeypatch, user_input, default, expected):
inputs = iter([user_input])
monkeypatch.setattr(builtins, "input", lambda _: next(inputs))
res = output.prompt_confirm("Continue?", default=default)
assert res is expected
def test_prompt_confirm_invalid_then_yes(monkeypatch, capsys):
inputs = iter(["maybe", "y"])
monkeypatch.setattr(builtins, "input", lambda _: next(inputs))
res = output.prompt_confirm("Continue?")
assert res is True
out = capsys.readouterr().out
assert "Invalid input." in out
+129
View File
@@ -0,0 +1,129 @@
import json
from pathlib import Path
import pytest
from decman.core.store import Store
def test_store_initially_empty_when_file_missing(tmp_path: Path) -> None:
path = tmp_path / "store.json"
assert not path.exists()
store = Store(path)
assert store.get("missing") is None
with pytest.raises(KeyError):
_ = store["missing"]
def test_store_loads_existing_file(tmp_path: Path) -> None:
path = tmp_path / "store.json"
original = {"foo": "bar", "number": 123}
path.write_text(json.dumps(original), encoding="utf-8")
store = Store(path)
assert store["foo"] == "bar"
assert store["number"] == 123
# underlying representation is dict-like
assert json.loads(path.read_text(encoding="utf-8")) == original
def test_setitem_and_getitem_roundtrip(tmp_path: Path) -> None:
path = tmp_path / "store.json"
store = Store(path)
store["foo"] = "bar"
store["number"] = 123
assert store["foo"] == "bar"
assert store["number"] == 123
def test_get_with_default(tmp_path: Path) -> None:
path = tmp_path / "store.json"
store = Store(path)
store["present"] = "value"
assert store.get("present") == "value"
assert store.get("missing") is None
assert store.get("missing", "default") == "default"
def test_save_creates_parent_directory_and_persists(tmp_path: Path) -> None:
# use nested directory to ensure parent creation is exercised
path = tmp_path / "nested" / "store.json"
store = Store(path)
store["foo"] = "bar"
store.save()
assert path.is_file()
data = json.loads(path.read_text(encoding="utf-8"))
assert data == {"foo": "bar"}
def test_context_manager_saves_on_normal_exit(tmp_path: Path) -> None:
path = tmp_path / "store.json"
with Store(path) as store:
store["foo"] = "bar"
store["number"] = 123
assert path.is_file()
data = json.loads(path.read_text(encoding="utf-8"))
assert data == {"foo": "bar", "number": 123}
def test_context_manager_saves_even_on_exception(tmp_path: Path) -> None:
path = tmp_path / "store.json"
with pytest.raises(RuntimeError):
with Store(path) as store:
store["foo"] = "bar"
raise RuntimeError("boom")
# file should still be written despite the exception
assert path.is_file()
data = json.loads(path.read_text(encoding="utf-8"))
assert data == {"foo": "bar"}
def test_repr_matches_underlying_dict(tmp_path: Path) -> None:
path = tmp_path / "store.json"
store = Store(path)
store["foo"] = "bar"
store["number"] = 123
expected = repr({"foo": "bar", "number": 123})
assert repr(store) == expected
def test_store_persists_sets(tmp_path: Path) -> None:
path = tmp_path / "store.json"
# initial write with sets
store = Store(path)
store["units"] = {"a.service", "b.service"}
store["user_units"] = {"alice": {"u1.service", "u2.service"}}
store.save()
# raw JSON should be set-encoded, not fail json.dump
raw = json.loads(path.read_text(encoding="utf-8"))
assert raw["units"]["__type__"] == "set"
assert set(raw["units"]["items"]) == {"a.service", "b.service"}
assert raw["user_units"]["alice"]["__type__"] == "set"
assert set(raw["user_units"]["alice"]["items"]) == {"u1.service", "u2.service"}
# reloading via Store must restore actual set objects
reloaded = Store(path)
assert reloaded["units"] == {"a.service", "b.service"}
assert isinstance(reloaded["units"], set)
assert reloaded["user_units"]["alice"] == {"u1.service", "u2.service"}
assert isinstance(reloaded["user_units"]["alice"], set)
+41
View File
@@ -0,0 +1,41 @@
import typing
import pytest
import decman
def test_sh_calls_prg_with_sh_command(monkeypatch: pytest.MonkeyPatch):
calls: dict[str, typing.Any] = {}
def fake_prg(
cmd,
user=None,
env_overrides=None,
mimic_login=False,
pty=True,
check=True,
):
calls["prg"] = (cmd, user, env_overrides, mimic_login, pty, check)
return "output-from-prg"
monkeypatch.setattr(decman, "prg", fake_prg)
out = decman.sh(
"echo test",
user="bob",
env_overrides={"X": "1"},
mimic_login=True,
pty=False,
check=False,
)
assert out == "output-from-prg"
cmd, user, env_overrides, mimic_login, pty, check = calls["prg"]
assert cmd == ["/bin/sh", "-c", "echo test"]
assert user == "bob"
assert env_overrides == {"X": "1"}
assert mimic_login is True
assert pty is False
assert check is False
+40
View File
@@ -0,0 +1,40 @@
from decman.core.module import Module
from decman.plugins import run_methods_with_attribute
def mark(attr):
attr.__flag__ = True
return attr
def test_runs_marked_method_and_returns_value():
class M(Module):
@mark
def foo(self):
return 123
m = M("m")
assert run_methods_with_attribute(m, "__flag__") == [123]
def test_runs_marked_methods_and_returns_value():
class M(Module):
@mark
def foo(self):
return 123
@mark
def bar(self):
return 321
m = M("m")
assert run_methods_with_attribute(m, "__flag__") == [321, 123]
def test_returns_none_if_no_method_has_attribute():
class M(Module):
def foo(self):
return 1
m = M("m")
assert run_methods_with_attribute(m, "__flag__") == []
-106
View File
@@ -1,106 +0,0 @@
# pylint: disable=missing-module-docstring,missing-class-docstring,missing-function-docstring
import unittest
from decman.error import UserFacingError
from decman.lib import Pacman, Store
from decman.lib.fpm import ForeignPackageManager, DepGraph, ForeignPackage, ExtendedPackageSearch
class TestVersionComparisons(unittest.TestCase):
def setUp(self):
pacman = Pacman()
self.pm = ForeignPackageManager(Store(), pacman,
ExtendedPackageSearch(pacman))
def test_should_upgrade_package_returns_true_on_newer_version(self):
self.assertTrue(
self.pm.should_upgrade_package("test", "0.1.9", "0.2.0"))
def test_should_upgrade_package_returns_false_on_older_version(self):
self.assertFalse(
self.pm.should_upgrade_package("test", "0.1.9", "0.1.8"))
def test_should_upgrade_package_returns_false_on_same_version(self):
self.assertFalse(
self.pm.should_upgrade_package("test", "0.1.9", "0.1.9"))
def test_should_upgrade_package_returns_true_on_devel(self):
self.assertTrue(
self.pm.should_upgrade_package("test-git",
"0",
"0",
upgrade_devel=True))
class TestDepGraph(unittest.TestCase):
def test_add_dependency(self):
graph = DepGraph()
graph.add_requirement("A", None)
graph.add_requirement("B1", "A")
graph.add_requirement("B2", "A")
graph.add_requirement("C", "B1")
self.assertIn("B1", graph.package_nodes["A"].children)
self.assertIn("B2", graph.package_nodes["A"].children)
self.assertIn("C", graph.package_nodes["B1"].children)
def test_cyclic_dep_fails(self):
graph = DepGraph()
graph.add_requirement("A", None)
graph.add_requirement("B", "A")
graph.add_requirement("C", "B")
with self.assertRaises(UserFacingError):
graph.add_requirement("A", "C")
def test_get_and_remove_outer_deps(self):
graph = DepGraph()
graph.add_requirement("A", None)
graph.add_requirement("V", None)
graph.add_requirement("B1", "A")
graph.add_requirement("B2", "A")
graph.add_requirement("B3", "A")
graph.add_requirement("B1", "B2")
graph.add_requirement("C1", "B1")
graph.add_requirement("C2", "B1")
graph.add_requirement("D", "C1")
graph.add_requirement("C2", "D")
v = ForeignPackage("V")
a = ForeignPackage("A")
a.add_foreign_dependency_packages(["B1", "B2", "B3", "C1", "C2", "D"])
b1 = ForeignPackage("B1")
b1.add_foreign_dependency_packages(["C1", "C2", "D"])
b2 = ForeignPackage("B2")
b2.add_foreign_dependency_packages(["B1", "C1", "C2", "D"])
b3 = ForeignPackage("B3")
c1 = ForeignPackage("C1")
c1.add_foreign_dependency_packages(["D", "C2"])
c2 = ForeignPackage("C2")
d = ForeignPackage("D")
d.add_foreign_dependency_packages(["C2"])
self.assertCountEqual(graph.get_and_remove_outer_dep_pkgs(),
[c2, b3, v])
self.assertCountEqual(graph.get_and_remove_outer_dep_pkgs(), [d])
self.assertCountEqual(graph.get_and_remove_outer_dep_pkgs(), [c1])
self.assertCountEqual(graph.get_and_remove_outer_dep_pkgs(), [b1])
self.assertCountEqual(graph.get_and_remove_outer_dep_pkgs(), [b2])
self.assertCountEqual(graph.get_and_remove_outer_dep_pkgs(), [a])
self.assertCountEqual(graph.get_and_remove_outer_dep_pkgs(), [])
-266
View File
@@ -1,266 +0,0 @@
# pylint: disable=missing-module-docstring,missing-class-docstring,missing-function-docstring
import unittest
from decman.lib import Source, Store
from decman import UserPackage, Module
class ExistingTestModule(Module):
def __init__(self):
self.on_enable_executed = False
self.on_disable_executed = False
self.after_update_executed = False
self.after_version_change_executed = False
super().__init__("Existing", True, "1")
def on_enable(self):
self.on_enable_executed = True
def on_disable(self):
self.on_disable_executed = True
def after_update(self):
self.after_update_executed = True
def after_version_change(self):
self.after_version_change_executed = True
class ExistingChangedVersionTestModule(Module):
def __init__(self):
self.on_enable_executed = False
self.on_disable_executed = False
self.after_update_executed = False
self.after_version_change_executed = False
super().__init__("ExistingChanged", True, "2")
def on_enable(self):
self.on_enable_executed = True
def on_disable(self):
self.on_disable_executed = True
def after_update(self):
self.after_update_executed = True
def after_version_change(self):
self.after_version_change_executed = True
class EnabledTestModule(Module):
def __init__(self):
self.on_enable_executed = False
self.on_disable_executed = False
self.after_update_executed = False
self.after_version_change_executed = False
super().__init__("Enabled", True, "1")
def on_enable(self):
self.on_enable_executed = True
def on_disable(self):
self.on_disable_executed = True
def after_update(self):
self.after_update_executed = True
def after_version_change(self):
self.after_version_change_executed = True
def pacman_packages(self) -> list[str]:
return ["M_p1", "M_p2", "M_p3"]
def systemd_user_units(self) -> dict[str, list[str]]:
return {"muser": ["M_u1.service"]}
class DisabledTestModule(Module):
def __init__(self):
self.on_enable_executed = False
self.on_disable_executed = False
self.after_update_executed = False
self.after_version_change_executed = False
super().__init__("Disabled", False, "1")
def on_enable(self):
self.on_enable_executed = True
def on_disable(self):
self.on_disable_executed = True
def after_update(self):
self.after_update_executed = True
def after_version_change(self):
self.after_version_change_executed = True
def aur_packages(self) -> list[str]:
return ["M_A1", "M_A2", "M_A3"]
def systemd_units(self) -> list[str]:
return ["M_1.service"]
class TestSource(unittest.TestCase):
def setUp(self):
self.disabled_module = DisabledTestModule()
self.enabled_module = EnabledTestModule()
self.existing_module = ExistingTestModule()
self.existing_module_changed = ExistingChangedVersionTestModule()
modules = {
self.enabled_module,
self.disabled_module,
self.existing_module,
self.existing_module_changed,
}
source = Source(
pacman_packages={"p1", "p2", "p3"},
aur_packages={"A1", "A2", "A3"},
user_packages={
UserPackage(
pkgname="U1",
version="1",
dependencies=["d1"],
git_url="/am/url/yes",
),
UserPackage(
pkgname="U2",
version="1",
dependencies=["d2"],
git_url="/am/url/yes",
)
},
ignored_packages={"i1", "i2"},
systemd_units={"1.service", "2.timer"},
systemd_user_units={"user": {"u1.service", "u2.timer"}},
modules=modules,
files={},
directories={},
)
store = Store()
store.enabled_systemd_units.extend(
["1.service", "3.service", "M_1.service"])
store.add_enabled_user_systemd_unit("user", "u1.service")
store.add_enabled_user_systemd_unit("user", "u3.service")
store.enabled_modules = {
"Existing": "1",
"ExistingChanged": "1",
"Disabled": "1",
}
store.created_files = ["/test/file1", "/test/file2", "/test/file3"]
currently_installed_packages = [
"p1",
"p2",
"p4",
"A2",
"A3",
"A4",
"U1",
"i1",
"M_p3",
"M_A1",
"M_A2",
]
self.source = source
self.store = store
self.currently_installed_packages = currently_installed_packages
def test_all_enabled_modules(self):
enabled_modules = [("Enabled", "1"), ("Existing", "1"),
("ExistingChanged", "2")]
self.assertCountEqual(self.source.all_enabled_modules(),
enabled_modules)
def test_files_to_remove(self):
created_files = ["/test/file1", "/test/file4"]
self.assertCountEqual(
self.source.files_to_remove(self.store, created_files),
["/test/file2", "/test/file3"])
def test_after_update_executed(self):
self.source.run_after_update()
self.assertTrue(self.enabled_module.after_update_executed)
self.assertTrue(self.existing_module.after_update_executed)
self.assertTrue(self.existing_module_changed.after_update_executed)
self.assertFalse(self.disabled_module.after_update_executed)
def test_after_version_change_executed(self):
self.source.run_after_version_change(self.store)
self.assertTrue(self.enabled_module.after_version_change_executed)
self.assertTrue(
self.existing_module_changed.after_version_change_executed)
self.assertFalse(self.existing_module.after_version_change_executed)
self.assertFalse(self.disabled_module.after_version_change_executed)
def test_on_enable_executed(self):
self.source.run_on_enable(self.store)
self.assertTrue(self.enabled_module.on_enable_executed)
self.assertFalse(self.disabled_module.on_enable_executed)
self.assertFalse(self.existing_module.on_enable_executed)
self.assertFalse(self.existing_module_changed.on_enable_executed)
def test_on_disable_executed(self):
self.source.run_on_disable(self.store)
self.assertTrue(self.disabled_module.on_disable_executed)
self.assertFalse(self.enabled_module.on_disable_executed)
self.assertFalse(self.existing_module.on_disable_executed)
self.assertFalse(self.existing_module_changed.on_disable_executed)
def test_units_to_enable(self):
self.assertCountEqual(
self.source.units_to_enable(self.store),
["2.timer"],
)
def test_units_to_disable(self):
self.assertCountEqual(
self.source.units_to_disable(self.store),
["3.service", "M_1.service"],
)
def test_user_units_to_enable(self):
self.assertDictEqual(
self.source.user_units_to_enable(self.store),
{
"user": ["u2.timer"],
"muser": ["M_u1.service"]
},
)
def test_user_units_to_disable(self):
self.assertDictEqual(
self.source.user_units_to_disable(self.store),
{"user": ["u3.service"]},
)
def test_pacman_packages_to_install(self):
self.assertCountEqual(
self.source.pacman_packages_to_install(
self.currently_installed_packages),
["p3", "M_p1", "M_p2"],
)
def test_foreign_packages_to_install(self):
self.assertCountEqual(
self.source.foreign_packages_to_install(
self.currently_installed_packages),
["A1", "U2"],
)
def test_packages_to_remove(self):
self.assertCountEqual(
self.source.packages_to_remove(self.currently_installed_packages),
["p4", "A4", "M_A1", "M_A2"],
)
Generated
+286
View File
@@ -0,0 +1,286 @@
version = 1
revision = 3
requires-python = ">=3.13"
[manifest]
members = [
"decman",
"decman-flatpak",
"decman-pacman",
"decman-systemd",
]
[[package]]
name = "certifi"
version = "2025.11.12"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/a2/8c/58f469717fa48465e4a50c014a0400602d3c437d7c0c468e17ada824da3a/certifi-2025.11.12.tar.gz", hash = "sha256:d8ab5478f2ecd78af242878415affce761ca6bc54a22a27e026d7c25357c3316", size = 160538, upload-time = "2025-11-12T02:54:51.517Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/70/7d/9bc192684cea499815ff478dfcdc13835ddf401365057044fb721ec6bddb/certifi-2025.11.12-py3-none-any.whl", hash = "sha256:97de8790030bbd5c2d96b7ec782fc2f7820ef8dba6db909ccf95449f2d062d4b", size = 159438, upload-time = "2025-11-12T02:54:49.735Z" },
]
[[package]]
name = "charset-normalizer"
version = "3.4.4"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/13/69/33ddede1939fdd074bce5434295f38fae7136463422fe4fd3e0e89b98062/charset_normalizer-3.4.4.tar.gz", hash = "sha256:94537985111c35f28720e43603b8e7b43a6ecfb2ce1d3058bbe955b73404e21a", size = 129418, upload-time = "2025-10-14T04:42:32.879Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/97/45/4b3a1239bbacd321068ea6e7ac28875b03ab8bc0aa0966452db17cd36714/charset_normalizer-3.4.4-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:e1f185f86a6f3403aa2420e815904c67b2f9ebc443f045edd0de921108345794", size = 208091, upload-time = "2025-10-14T04:41:13.346Z" },
{ url = "https://files.pythonhosted.org/packages/7d/62/73a6d7450829655a35bb88a88fca7d736f9882a27eacdca2c6d505b57e2e/charset_normalizer-3.4.4-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6b39f987ae8ccdf0d2642338faf2abb1862340facc796048b604ef14919e55ed", size = 147936, upload-time = "2025-10-14T04:41:14.461Z" },
{ url = "https://files.pythonhosted.org/packages/89/c5/adb8c8b3d6625bef6d88b251bbb0d95f8205831b987631ab0c8bb5d937c2/charset_normalizer-3.4.4-cp313-cp313-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:3162d5d8ce1bb98dd51af660f2121c55d0fa541b46dff7bb9b9f86ea1d87de72", size = 144180, upload-time = "2025-10-14T04:41:15.588Z" },
{ url = "https://files.pythonhosted.org/packages/91/ed/9706e4070682d1cc219050b6048bfd293ccf67b3d4f5a4f39207453d4b99/charset_normalizer-3.4.4-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:81d5eb2a312700f4ecaa977a8235b634ce853200e828fbadf3a9c50bab278328", size = 161346, upload-time = "2025-10-14T04:41:16.738Z" },
{ url = "https://files.pythonhosted.org/packages/d5/0d/031f0d95e4972901a2f6f09ef055751805ff541511dc1252ba3ca1f80cf5/charset_normalizer-3.4.4-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:5bd2293095d766545ec1a8f612559f6b40abc0eb18bb2f5d1171872d34036ede", size = 158874, upload-time = "2025-10-14T04:41:17.923Z" },
{ url = "https://files.pythonhosted.org/packages/f5/83/6ab5883f57c9c801ce5e5677242328aa45592be8a00644310a008d04f922/charset_normalizer-3.4.4-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a8a8b89589086a25749f471e6a900d3f662d1d3b6e2e59dcecf787b1cc3a1894", size = 153076, upload-time = "2025-10-14T04:41:19.106Z" },
{ url = "https://files.pythonhosted.org/packages/75/1e/5ff781ddf5260e387d6419959ee89ef13878229732732ee73cdae01800f2/charset_normalizer-3.4.4-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:bc7637e2f80d8530ee4a78e878bce464f70087ce73cf7c1caf142416923b98f1", size = 150601, upload-time = "2025-10-14T04:41:20.245Z" },
{ url = "https://files.pythonhosted.org/packages/d7/57/71be810965493d3510a6ca79b90c19e48696fb1ff964da319334b12677f0/charset_normalizer-3.4.4-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:f8bf04158c6b607d747e93949aa60618b61312fe647a6369f88ce2ff16043490", size = 150376, upload-time = "2025-10-14T04:41:21.398Z" },
{ url = "https://files.pythonhosted.org/packages/e5/d5/c3d057a78c181d007014feb7e9f2e65905a6c4ef182c0ddf0de2924edd65/charset_normalizer-3.4.4-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:554af85e960429cf30784dd47447d5125aaa3b99a6f0683589dbd27e2f45da44", size = 144825, upload-time = "2025-10-14T04:41:22.583Z" },
{ url = "https://files.pythonhosted.org/packages/e6/8c/d0406294828d4976f275ffbe66f00266c4b3136b7506941d87c00cab5272/charset_normalizer-3.4.4-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:74018750915ee7ad843a774364e13a3db91682f26142baddf775342c3f5b1133", size = 162583, upload-time = "2025-10-14T04:41:23.754Z" },
{ url = "https://files.pythonhosted.org/packages/d7/24/e2aa1f18c8f15c4c0e932d9287b8609dd30ad56dbe41d926bd846e22fb8d/charset_normalizer-3.4.4-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:c0463276121fdee9c49b98908b3a89c39be45d86d1dbaa22957e38f6321d4ce3", size = 150366, upload-time = "2025-10-14T04:41:25.27Z" },
{ url = "https://files.pythonhosted.org/packages/e4/5b/1e6160c7739aad1e2df054300cc618b06bf784a7a164b0f238360721ab86/charset_normalizer-3.4.4-cp313-cp313-musllinux_1_2_s390x.whl", hash = "sha256:362d61fd13843997c1c446760ef36f240cf81d3ebf74ac62652aebaf7838561e", size = 160300, upload-time = "2025-10-14T04:41:26.725Z" },
{ url = "https://files.pythonhosted.org/packages/7a/10/f882167cd207fbdd743e55534d5d9620e095089d176d55cb22d5322f2afd/charset_normalizer-3.4.4-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:9a26f18905b8dd5d685d6d07b0cdf98a79f3c7a918906af7cc143ea2e164c8bc", size = 154465, upload-time = "2025-10-14T04:41:28.322Z" },
{ url = "https://files.pythonhosted.org/packages/89/66/c7a9e1b7429be72123441bfdbaf2bc13faab3f90b933f664db506dea5915/charset_normalizer-3.4.4-cp313-cp313-win32.whl", hash = "sha256:9b35f4c90079ff2e2edc5b26c0c77925e5d2d255c42c74fdb70fb49b172726ac", size = 99404, upload-time = "2025-10-14T04:41:29.95Z" },
{ url = "https://files.pythonhosted.org/packages/c4/26/b9924fa27db384bdcd97ab83b4f0a8058d96ad9626ead570674d5e737d90/charset_normalizer-3.4.4-cp313-cp313-win_amd64.whl", hash = "sha256:b435cba5f4f750aa6c0a0d92c541fb79f69a387c91e61f1795227e4ed9cece14", size = 107092, upload-time = "2025-10-14T04:41:31.188Z" },
{ url = "https://files.pythonhosted.org/packages/af/8f/3ed4bfa0c0c72a7ca17f0380cd9e4dd842b09f664e780c13cff1dcf2ef1b/charset_normalizer-3.4.4-cp313-cp313-win_arm64.whl", hash = "sha256:542d2cee80be6f80247095cc36c418f7bddd14f4a6de45af91dfad36d817bba2", size = 100408, upload-time = "2025-10-14T04:41:32.624Z" },
{ url = "https://files.pythonhosted.org/packages/2a/35/7051599bd493e62411d6ede36fd5af83a38f37c4767b92884df7301db25d/charset_normalizer-3.4.4-cp314-cp314-macosx_10_13_universal2.whl", hash = "sha256:da3326d9e65ef63a817ecbcc0df6e94463713b754fe293eaa03da99befb9a5bd", size = 207746, upload-time = "2025-10-14T04:41:33.773Z" },
{ url = "https://files.pythonhosted.org/packages/10/9a/97c8d48ef10d6cd4fcead2415523221624bf58bcf68a802721a6bc807c8f/charset_normalizer-3.4.4-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:8af65f14dc14a79b924524b1e7fffe304517b2bff5a58bf64f30b98bbc5079eb", size = 147889, upload-time = "2025-10-14T04:41:34.897Z" },
{ url = "https://files.pythonhosted.org/packages/10/bf/979224a919a1b606c82bd2c5fa49b5c6d5727aa47b4312bb27b1734f53cd/charset_normalizer-3.4.4-cp314-cp314-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:74664978bb272435107de04e36db5a9735e78232b85b77d45cfb38f758efd33e", size = 143641, upload-time = "2025-10-14T04:41:36.116Z" },
{ url = "https://files.pythonhosted.org/packages/ba/33/0ad65587441fc730dc7bd90e9716b30b4702dc7b617e6ba4997dc8651495/charset_normalizer-3.4.4-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:752944c7ffbfdd10c074dc58ec2d5a8a4cd9493b314d367c14d24c17684ddd14", size = 160779, upload-time = "2025-10-14T04:41:37.229Z" },
{ url = "https://files.pythonhosted.org/packages/67/ed/331d6b249259ee71ddea93f6f2f0a56cfebd46938bde6fcc6f7b9a3d0e09/charset_normalizer-3.4.4-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:d1f13550535ad8cff21b8d757a3257963e951d96e20ec82ab44bc64aeb62a191", size = 159035, upload-time = "2025-10-14T04:41:38.368Z" },
{ url = "https://files.pythonhosted.org/packages/67/ff/f6b948ca32e4f2a4576aa129d8bed61f2e0543bf9f5f2b7fc3758ed005c9/charset_normalizer-3.4.4-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ecaae4149d99b1c9e7b88bb03e3221956f68fd6d50be2ef061b2381b61d20838", size = 152542, upload-time = "2025-10-14T04:41:39.862Z" },
{ url = "https://files.pythonhosted.org/packages/16/85/276033dcbcc369eb176594de22728541a925b2632f9716428c851b149e83/charset_normalizer-3.4.4-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:cb6254dc36b47a990e59e1068afacdcd02958bdcce30bb50cc1700a8b9d624a6", size = 149524, upload-time = "2025-10-14T04:41:41.319Z" },
{ url = "https://files.pythonhosted.org/packages/9e/f2/6a2a1f722b6aba37050e626530a46a68f74e63683947a8acff92569f979a/charset_normalizer-3.4.4-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:c8ae8a0f02f57a6e61203a31428fa1d677cbe50c93622b4149d5c0f319c1d19e", size = 150395, upload-time = "2025-10-14T04:41:42.539Z" },
{ url = "https://files.pythonhosted.org/packages/60/bb/2186cb2f2bbaea6338cad15ce23a67f9b0672929744381e28b0592676824/charset_normalizer-3.4.4-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:47cc91b2f4dd2833fddaedd2893006b0106129d4b94fdb6af1f4ce5a9965577c", size = 143680, upload-time = "2025-10-14T04:41:43.661Z" },
{ url = "https://files.pythonhosted.org/packages/7d/a5/bf6f13b772fbb2a90360eb620d52ed8f796f3c5caee8398c3b2eb7b1c60d/charset_normalizer-3.4.4-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:82004af6c302b5d3ab2cfc4cc5f29db16123b1a8417f2e25f9066f91d4411090", size = 162045, upload-time = "2025-10-14T04:41:44.821Z" },
{ url = "https://files.pythonhosted.org/packages/df/c5/d1be898bf0dc3ef9030c3825e5d3b83f2c528d207d246cbabe245966808d/charset_normalizer-3.4.4-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:2b7d8f6c26245217bd2ad053761201e9f9680f8ce52f0fcd8d0755aeae5b2152", size = 149687, upload-time = "2025-10-14T04:41:46.442Z" },
{ url = "https://files.pythonhosted.org/packages/a5/42/90c1f7b9341eef50c8a1cb3f098ac43b0508413f33affd762855f67a410e/charset_normalizer-3.4.4-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:799a7a5e4fb2d5898c60b640fd4981d6a25f1c11790935a44ce38c54e985f828", size = 160014, upload-time = "2025-10-14T04:41:47.631Z" },
{ url = "https://files.pythonhosted.org/packages/76/be/4d3ee471e8145d12795ab655ece37baed0929462a86e72372fd25859047c/charset_normalizer-3.4.4-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:99ae2cffebb06e6c22bdc25801d7b30f503cc87dbd283479e7b606f70aff57ec", size = 154044, upload-time = "2025-10-14T04:41:48.81Z" },
{ url = "https://files.pythonhosted.org/packages/b0/6f/8f7af07237c34a1defe7defc565a9bc1807762f672c0fde711a4b22bf9c0/charset_normalizer-3.4.4-cp314-cp314-win32.whl", hash = "sha256:f9d332f8c2a2fcbffe1378594431458ddbef721c1769d78e2cbc06280d8155f9", size = 99940, upload-time = "2025-10-14T04:41:49.946Z" },
{ url = "https://files.pythonhosted.org/packages/4b/51/8ade005e5ca5b0d80fb4aff72a3775b325bdc3d27408c8113811a7cbe640/charset_normalizer-3.4.4-cp314-cp314-win_amd64.whl", hash = "sha256:8a6562c3700cce886c5be75ade4a5db4214fda19fede41d9792d100288d8f94c", size = 107104, upload-time = "2025-10-14T04:41:51.051Z" },
{ url = "https://files.pythonhosted.org/packages/da/5f/6b8f83a55bb8278772c5ae54a577f3099025f9ade59d0136ac24a0df4bde/charset_normalizer-3.4.4-cp314-cp314-win_arm64.whl", hash = "sha256:de00632ca48df9daf77a2c65a484531649261ec9f25489917f09e455cb09ddb2", size = 100743, upload-time = "2025-10-14T04:41:52.122Z" },
{ url = "https://files.pythonhosted.org/packages/0a/4c/925909008ed5a988ccbb72dcc897407e5d6d3bd72410d69e051fc0c14647/charset_normalizer-3.4.4-py3-none-any.whl", hash = "sha256:7a32c560861a02ff789ad905a2fe94e3f840803362c84fecf1851cb4cf3dc37f", size = 53402, upload-time = "2025-10-14T04:42:31.76Z" },
]
[[package]]
name = "colorama"
version = "0.4.6"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/d8/53/6f443c9a4a8358a93a6792e2acffb9d9d5cb0a5cfd8802644b7b1c9a02e4/colorama-0.4.6.tar.gz", hash = "sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44", size = 27697, upload-time = "2022-10-25T02:36:22.414Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6", size = 25335, upload-time = "2022-10-25T02:36:20.889Z" },
]
[[package]]
name = "decman"
version = "1.2.0"
source = { editable = "." }
[package.optional-dependencies]
flatpak = [
{ name = "decman-flatpak" },
]
pacman = [
{ name = "decman-pacman" },
]
systemd = [
{ name = "decman-systemd" },
]
[package.dev-dependencies]
dev = [
{ name = "pytest" },
{ name = "ruff" },
]
[package.metadata]
requires-dist = [
{ name = "decman-flatpak", marker = "extra == 'flatpak'", editable = "plugins/decman-flatpak" },
{ name = "decman-pacman", marker = "extra == 'pacman'", editable = "plugins/decman-pacman" },
{ name = "decman-systemd", marker = "extra == 'systemd'", editable = "plugins/decman-systemd" },
]
provides-extras = ["pacman", "systemd", "flatpak"]
[package.metadata.requires-dev]
dev = [
{ name = "pytest", specifier = ">=8.4.2" },
{ name = "ruff", specifier = ">=0.14.9" },
]
[[package]]
name = "decman-flatpak"
version = "1.1.0"
source = { editable = "plugins/decman-flatpak" }
dependencies = [
{ name = "decman" },
]
[package.metadata]
requires-dist = [{ name = "decman", editable = "." }]
[[package]]
name = "decman-pacman"
version = "1.1.0"
source = { editable = "plugins/decman-pacman" }
dependencies = [
{ name = "decman" },
{ name = "pyalpm" },
{ name = "requests" },
]
[package.dev-dependencies]
dev = [
{ name = "pytest" },
{ name = "pytest-mock" },
]
[package.metadata]
requires-dist = [
{ name = "decman", editable = "." },
{ name = "pyalpm" },
{ name = "requests" },
]
[package.metadata.requires-dev]
dev = [
{ name = "pytest", specifier = ">=8.4.2" },
{ name = "pytest-mock", specifier = ">=3.15.1" },
]
[[package]]
name = "decman-systemd"
version = "1.1.0"
source = { editable = "plugins/decman-systemd" }
dependencies = [
{ name = "decman" },
]
[package.metadata]
requires-dist = [{ name = "decman", editable = "." }]
[[package]]
name = "idna"
version = "3.11"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/6f/6d/0703ccc57f3a7233505399edb88de3cbd678da106337b9fcde432b65ed60/idna-3.11.tar.gz", hash = "sha256:795dafcc9c04ed0c1fb032c2aa73654d8e8c5023a7df64a53f39190ada629902", size = 194582, upload-time = "2025-10-12T14:55:20.501Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/0e/61/66938bbb5fc52dbdf84594873d5b51fb1f7c7794e9c0f5bd885f30bc507b/idna-3.11-py3-none-any.whl", hash = "sha256:771a87f49d9defaf64091e6e6fe9c18d4833f140bd19464795bc32d966ca37ea", size = 71008, upload-time = "2025-10-12T14:55:18.883Z" },
]
[[package]]
name = "iniconfig"
version = "2.3.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/72/34/14ca021ce8e5dfedc35312d08ba8bf51fdd999c576889fc2c24cb97f4f10/iniconfig-2.3.0.tar.gz", hash = "sha256:c76315c77db068650d49c5b56314774a7804df16fee4402c1f19d6d15d8c4730", size = 20503, upload-time = "2025-10-18T21:55:43.219Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/cb/b1/3846dd7f199d53cb17f49cba7e651e9ce294d8497c8c150530ed11865bb8/iniconfig-2.3.0-py3-none-any.whl", hash = "sha256:f631c04d2c48c52b84d0d0549c99ff3859c98df65b3101406327ecc7d53fbf12", size = 7484, upload-time = "2025-10-18T21:55:41.639Z" },
]
[[package]]
name = "packaging"
version = "25.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/a1/d4/1fc4078c65507b51b96ca8f8c3ba19e6a61c8253c72794544580a7b6c24d/packaging-25.0.tar.gz", hash = "sha256:d443872c98d677bf60f6a1f2f8c1cb748e8fe762d2bf9d3148b5599295b0fc4f", size = 165727, upload-time = "2025-04-19T11:48:59.673Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/20/12/38679034af332785aac8774540895e234f4d07f7545804097de4b666afd8/packaging-25.0-py3-none-any.whl", hash = "sha256:29572ef2b1f17581046b3a2227d5c611fb25ec70ca1ba8554b24b0e69331a484", size = 66469, upload-time = "2025-04-19T11:48:57.875Z" },
]
[[package]]
name = "pluggy"
version = "1.6.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/f9/e2/3e91f31a7d2b083fe6ef3fa267035b518369d9511ffab804f839851d2779/pluggy-1.6.0.tar.gz", hash = "sha256:7dcc130b76258d33b90f61b658791dede3486c3e6bfb003ee5c9bfb396dd22f3", size = 69412, upload-time = "2025-05-15T12:30:07.975Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/54/20/4d324d65cc6d9205fabedc306948156824eb9f0ee1633355a8f7ec5c66bf/pluggy-1.6.0-py3-none-any.whl", hash = "sha256:e920276dd6813095e9377c0bc5566d94c932c33b27a3e3945d8389c374dd4746", size = 20538, upload-time = "2025-05-15T12:30:06.134Z" },
]
[[package]]
name = "pyalpm"
version = "0.10.12"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/fa/33/fb965c94a703b2f62741a52911446fe85e7effc677d54d1a8cd0033f4848/pyalpm-0.10.12.tar.gz", hash = "sha256:8c6cb4bbba819f99fe6c2547f25a861f68905d96729caf1ea15ba6c43d4f1127", size = 52295, upload-time = "2025-05-27T10:33:58.208Z" }
[[package]]
name = "pygments"
version = "2.19.2"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/b0/77/a5b8c569bf593b0140bde72ea885a803b82086995367bf2037de0159d924/pygments-2.19.2.tar.gz", hash = "sha256:636cb2477cec7f8952536970bc533bc43743542f70392ae026374600add5b887", size = 4968631, upload-time = "2025-06-21T13:39:12.283Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/c7/21/705964c7812476f378728bdf590ca4b771ec72385c533964653c68e86bdc/pygments-2.19.2-py3-none-any.whl", hash = "sha256:86540386c03d588bb81d44bc3928634ff26449851e99741617ecb9037ee5ec0b", size = 1225217, upload-time = "2025-06-21T13:39:07.939Z" },
]
[[package]]
name = "pytest"
version = "9.0.2"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "colorama", marker = "sys_platform == 'win32'" },
{ name = "iniconfig" },
{ name = "packaging" },
{ name = "pluggy" },
{ name = "pygments" },
]
sdist = { url = "https://files.pythonhosted.org/packages/d1/db/7ef3487e0fb0049ddb5ce41d3a49c235bf9ad299b6a25d5780a89f19230f/pytest-9.0.2.tar.gz", hash = "sha256:75186651a92bd89611d1d9fc20f0b4345fd827c41ccd5c299a868a05d70edf11", size = 1568901, upload-time = "2025-12-06T21:30:51.014Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/3b/ab/b3226f0bd7cdcf710fbede2b3548584366da3b19b5021e74f5bde2a8fa3f/pytest-9.0.2-py3-none-any.whl", hash = "sha256:711ffd45bf766d5264d487b917733b453d917afd2b0ad65223959f59089f875b", size = 374801, upload-time = "2025-12-06T21:30:49.154Z" },
]
[[package]]
name = "pytest-mock"
version = "3.15.1"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "pytest" },
]
sdist = { url = "https://files.pythonhosted.org/packages/68/14/eb014d26be205d38ad5ad20d9a80f7d201472e08167f0bb4361e251084a9/pytest_mock-3.15.1.tar.gz", hash = "sha256:1849a238f6f396da19762269de72cb1814ab44416fa73a8686deac10b0d87a0f", size = 34036, upload-time = "2025-09-16T16:37:27.081Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/5a/cc/06253936f4a7fa2e0f48dfe6d851d9c56df896a9ab09ac019d70b760619c/pytest_mock-3.15.1-py3-none-any.whl", hash = "sha256:0a25e2eb88fe5168d535041d09a4529a188176ae608a6d249ee65abc0949630d", size = 10095, upload-time = "2025-09-16T16:37:25.734Z" },
]
[[package]]
name = "requests"
version = "2.32.5"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "certifi" },
{ name = "charset-normalizer" },
{ name = "idna" },
{ name = "urllib3" },
]
sdist = { url = "https://files.pythonhosted.org/packages/c9/74/b3ff8e6c8446842c3f5c837e9c3dfcfe2018ea6ecef224c710c85ef728f4/requests-2.32.5.tar.gz", hash = "sha256:dbba0bac56e100853db0ea71b82b4dfd5fe2bf6d3754a8893c3af500cec7d7cf", size = 134517, upload-time = "2025-08-18T20:46:02.573Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/1e/db/4254e3eabe8020b458f1a747140d32277ec7a271daf1d235b70dc0b4e6e3/requests-2.32.5-py3-none-any.whl", hash = "sha256:2462f94637a34fd532264295e186976db0f5d453d1cdd31473c85a6a161affb6", size = 64738, upload-time = "2025-08-18T20:46:00.542Z" },
]
[[package]]
name = "ruff"
version = "0.14.9"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/f6/1b/ab712a9d5044435be8e9a2beb17cbfa4c241aa9b5e4413febac2a8b79ef2/ruff-0.14.9.tar.gz", hash = "sha256:35f85b25dd586381c0cc053f48826109384c81c00ad7ef1bd977bfcc28119d5b", size = 5809165, upload-time = "2025-12-11T21:39:47.381Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/b8/1c/d1b1bba22cffec02351c78ab9ed4f7d7391876e12720298448b29b7229c1/ruff-0.14.9-py3-none-linux_armv6l.whl", hash = "sha256:f1ec5de1ce150ca6e43691f4a9ef5c04574ad9ca35c8b3b0e18877314aba7e75", size = 13576541, upload-time = "2025-12-11T21:39:14.806Z" },
{ url = "https://files.pythonhosted.org/packages/94/ab/ffe580e6ea1fca67f6337b0af59fc7e683344a43642d2d55d251ff83ceae/ruff-0.14.9-py3-none-macosx_10_12_x86_64.whl", hash = "sha256:ed9d7417a299fc6030b4f26333bf1117ed82a61ea91238558c0268c14e00d0c2", size = 13779363, upload-time = "2025-12-11T21:39:20.29Z" },
{ url = "https://files.pythonhosted.org/packages/7d/f8/2be49047f929d6965401855461e697ab185e1a6a683d914c5c19c7962d9e/ruff-0.14.9-py3-none-macosx_11_0_arm64.whl", hash = "sha256:d5dc3473c3f0e4a1008d0ef1d75cee24a48e254c8bed3a7afdd2b4392657ed2c", size = 12925292, upload-time = "2025-12-11T21:39:38.757Z" },
{ url = "https://files.pythonhosted.org/packages/9e/e9/08840ff5127916bb989c86f18924fd568938b06f58b60e206176f327c0fe/ruff-0.14.9-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:84bf7c698fc8f3cb8278830fb6b5a47f9bcc1ed8cb4f689b9dd02698fa840697", size = 13362894, upload-time = "2025-12-11T21:39:02.524Z" },
{ url = "https://files.pythonhosted.org/packages/31/1c/5b4e8e7750613ef43390bb58658eaf1d862c0cc3352d139cd718a2cea164/ruff-0.14.9-py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:aa733093d1f9d88a5d98988d8834ef5d6f9828d03743bf5e338bf980a19fce27", size = 13311482, upload-time = "2025-12-11T21:39:17.51Z" },
{ url = "https://files.pythonhosted.org/packages/5b/3a/459dce7a8cb35ba1ea3e9c88f19077667a7977234f3b5ab197fad240b404/ruff-0.14.9-py3-none-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:6a1cfb04eda979b20c8c19550c8b5f498df64ff8da151283311ce3199e8b3648", size = 14016100, upload-time = "2025-12-11T21:39:41.948Z" },
{ url = "https://files.pythonhosted.org/packages/a6/31/f064f4ec32524f9956a0890fc6a944e5cf06c63c554e39957d208c0ffc45/ruff-0.14.9-py3-none-manylinux_2_17_ppc64.manylinux2014_ppc64.whl", hash = "sha256:1e5cb521e5ccf0008bd74d5595a4580313844a42b9103b7388eca5a12c970743", size = 15477729, upload-time = "2025-12-11T21:39:23.279Z" },
{ url = "https://files.pythonhosted.org/packages/7a/6d/f364252aad36ccd443494bc5f02e41bf677f964b58902a17c0b16c53d890/ruff-0.14.9-py3-none-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:cd429a8926be6bba4befa8cdcf3f4dd2591c413ea5066b1e99155ed245ae42bb", size = 15122386, upload-time = "2025-12-11T21:39:33.125Z" },
{ url = "https://files.pythonhosted.org/packages/20/02/e848787912d16209aba2799a4d5a1775660b6a3d0ab3944a4ccc13e64a02/ruff-0.14.9-py3-none-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:ab208c1b7a492e37caeaf290b1378148f75e13c2225af5d44628b95fd7834273", size = 14497124, upload-time = "2025-12-11T21:38:59.33Z" },
{ url = "https://files.pythonhosted.org/packages/f3/51/0489a6a5595b7760b5dbac0dd82852b510326e7d88d51dbffcd2e07e3ff3/ruff-0.14.9-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:72034534e5b11e8a593f517b2f2f2b273eb68a30978c6a2d40473ad0aaa4cb4a", size = 14195343, upload-time = "2025-12-11T21:39:44.866Z" },
{ url = "https://files.pythonhosted.org/packages/f6/53/3bb8d2fa73e4c2f80acc65213ee0830fa0c49c6479313f7a68a00f39e208/ruff-0.14.9-py3-none-manylinux_2_31_riscv64.whl", hash = "sha256:712ff04f44663f1b90a1195f51525836e3413c8a773574a7b7775554269c30ed", size = 14346425, upload-time = "2025-12-11T21:39:05.927Z" },
{ url = "https://files.pythonhosted.org/packages/ad/04/bdb1d0ab876372da3e983896481760867fc84f969c5c09d428e8f01b557f/ruff-0.14.9-py3-none-musllinux_1_2_aarch64.whl", hash = "sha256:a111fee1db6f1d5d5810245295527cda1d367c5aa8f42e0fca9a78ede9b4498b", size = 13258768, upload-time = "2025-12-11T21:39:08.691Z" },
{ url = "https://files.pythonhosted.org/packages/40/d9/8bf8e1e41a311afd2abc8ad12be1b6c6c8b925506d9069b67bb5e9a04af3/ruff-0.14.9-py3-none-musllinux_1_2_armv7l.whl", hash = "sha256:8769efc71558fecc25eb295ddec7d1030d41a51e9dcf127cbd63ec517f22d567", size = 13326939, upload-time = "2025-12-11T21:39:53.842Z" },
{ url = "https://files.pythonhosted.org/packages/f4/56/a213fa9edb6dd849f1cfbc236206ead10913693c72a67fb7ddc1833bf95d/ruff-0.14.9-py3-none-musllinux_1_2_i686.whl", hash = "sha256:347e3bf16197e8a2de17940cd75fd6491e25c0aa7edf7d61aa03f146a1aa885a", size = 13578888, upload-time = "2025-12-11T21:39:35.988Z" },
{ url = "https://files.pythonhosted.org/packages/33/09/6a4a67ffa4abae6bf44c972a4521337ffce9cbc7808faadede754ef7a79c/ruff-0.14.9-py3-none-musllinux_1_2_x86_64.whl", hash = "sha256:7715d14e5bccf5b660f54516558aa94781d3eb0838f8e706fb60e3ff6eff03a8", size = 14314473, upload-time = "2025-12-11T21:39:50.78Z" },
{ url = "https://files.pythonhosted.org/packages/12/0d/15cc82da5d83f27a3c6b04f3a232d61bc8c50d38a6cd8da79228e5f8b8d6/ruff-0.14.9-py3-none-win32.whl", hash = "sha256:df0937f30aaabe83da172adaf8937003ff28172f59ca9f17883b4213783df197", size = 13202651, upload-time = "2025-12-11T21:39:26.628Z" },
{ url = "https://files.pythonhosted.org/packages/32/f7/c78b060388eefe0304d9d42e68fab8cffd049128ec466456cef9b8d4f06f/ruff-0.14.9-py3-none-win_amd64.whl", hash = "sha256:c0b53a10e61df15a42ed711ec0bda0c582039cf6c754c49c020084c55b5b0bc2", size = 14702079, upload-time = "2025-12-11T21:39:11.954Z" },
{ url = "https://files.pythonhosted.org/packages/26/09/7a9520315decd2334afa65ed258fed438f070e31f05a2e43dd480a5e5911/ruff-0.14.9-py3-none-win_arm64.whl", hash = "sha256:8e821c366517a074046d92f0e9213ed1c13dbc5b37a7fc20b07f79b64d62cc84", size = 13744730, upload-time = "2025-12-11T21:39:29.659Z" },
]
[[package]]
name = "urllib3"
version = "2.6.2"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/1e/24/a2a2ed9addd907787d7aa0355ba36a6cadf1768b934c652ea78acbd59dcd/urllib3-2.6.2.tar.gz", hash = "sha256:016f9c98bb7e98085cb2b4b17b87d2c702975664e4f060c6532e64d1c1a5e797", size = 432930, upload-time = "2025-12-11T15:56:40.252Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/6d/b9/4095b668ea3678bf6a0af005527f39de12fb026516fb3df17495a733b7f8/urllib3-2.6.2-py3-none-any.whl", hash = "sha256:ec21cddfe7724fc7cb4ba4bea7aa8e2ef36f607a4bab81aa6ce42a13dc3f03dd", size = 131182, upload-time = "2025-12-11T15:56:38.584Z" },
]