Metadata-Version: 2.1
Name: voidup
Version: 0.1.2
Summary: Void repository and package management tools
License: AGPLv3
Author: Sumit Khanna
Author-email: sumit@penguindreasm.org
Requires-Python: >=3.12,<4.0
Classifier: License :: Other/Proprietary License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Requires-Dist: click (>=8.1.7,<9.0.0)
Requires-Dist: pytz (>=2024.2,<2025.0)
Requires-Dist: pyyaml (>=6.0.2,<7.0.0)
Requires-Dist: requests (>=2.32.3,<3.0.0)
Requires-Dist: rich (>=13.9.2,<14.0.0)
Requires-Dist: rsa (>=4.9,<5.0)
Requires-Dist: tzlocal (>=5.2,<6.0)
Description-Content-Type: text/markdown

# Tools for Updating Packages and Managing Custom Repositories on Void Linux

`voidup` is a tool for building custom package repositories for Void Linux. It can build template files from an overlay (either a local directory or a remote git repository). It uses overlayfs to merger several overlays together on top of the [void-packages](https://github.com/void-linux/void-packages) repository, so that custom packages can be built and maintained without creating custom forks and branches of the base repository.

**EARLY ALPHA**. Configuration subject to change.

## Installation

You can install the latest `voidup` from the [BPVL](https://bpvl.battlepenguin.com).

```shell
echo "repository=https://bpvl.battlepenguin.com" > /etc/xbps.d/bpvl.conf
xbps-install -S voidup
```

If you don't want to install the xbps package, you can also install `voidup` in a Python virtual environment from the official package repository:

```shell
python -m venv voidup-venv
source voidup-venv/bin/activate
pip install voidup --extra-index-url https://nexus.sumit.im/repository/bp-python/simple
```

## Usage

* `voidup update-system`: Runs a system update if outdated packages are found
* `voidup build-packages`: Builds packages defined by a build configuration

Why would you use `voidup update-system` over just calling `xbps-install -Syu`? `voidup` can be configured to log to either a file, or an instance of _loki-grafana_. Using _loki_ can help monitor updates across multiple system or clusters. 

## Configuration

If you install the `voidup` package, a default configuration will be available at `/etc/voidup/voidup.yaml`. The following configuration will log to both _loki-grafana_, and to a file. It enables _non-free_ packages that exist within the _void-packages_ repository, but which are not built by default. It will create a signature key if one does not exist, build the listed packages, and sign all of them with the generated key.

```yaml
user: voidup
schema: 0
logging:
  loki:
    url: https://example.com/api/prom/push
  file:
    filename: /var/log/voidup.log
    format: "%(asctime)s - %(name)s - %(levelname)s - %(message)s"    
repositories:
  allow_non_free: true
  build:
    lib_dir: /var/lib/voidup
    repos_dir: /var/lib/voidup/repos
    work_dir: /var/lib/voidup/overlayfs/work
    upper_dir: /var/lib/voidup/overlayfs/upper
    merged_dir: /var/lib/voidup/overlayfs/merged
  signing:
    pem_file: /var/lib/voidup/repository.pem
    signed_by: 'Bob Bear <bob@bob.bob>'
    sign_repos:
      - nonfree
      - BPVL
  overlays:
    BPVL:
      git: https://gitlab.com/djsumdog/BPVL
  build_sets:
    main:
      targets:
        - x86_64
        - x86_64-musl
        - aarch64
        - aarch64-musl
      packages:
        - supervisor
    nonfree:
      targets:
        - x86_64
      packages:
        - zoom
        - slack-desktop
        - google-chrome
```

The `build_sets` section determines which packages to build and for which architecture targets. In the above example, the non-free packages are made from precompiled binary packages, so we limit their targets to supported architectures. The build sets are run on the merged set of `void-packages` and all overlays.

The build output should be pulled from the _upper_dir_ in the above example. The _zoom_, _slack-desktop_ and _google-chrome_ packages will be located in `/var/lib/voidup/overlayfs/upper/hostdir/binpkgs/nonfree/` and _supervisor_ (which comes from the [BPVL overlay](https://gitlab.com/djsumdog/BPVL)) will be in `/var/lib/voidup/overlayfs/upper/hostdir/binpkgs/BPVL`.

**Do not publicly host non-free packages!** The above example is to show how you can build proprietary packages to host on your internal network.

The generated `xbps`, `xbps.sig2` and `<arch>-repodata` files can be copied to a web server for hosting your repository. When creating overlays, I recommend setting `repository=<your custom name>` to keep your custom packages separate from all the standard Void Linux packages. See the [BPVL repository](https://gitlab.com/djsumdog/BPVL) as an example.

## Live Build Outputs

The very first builds for any package will take a considerable amount of package bootstrapping. It may seem like `voidup` has stalled from the console. Setting `live_build_output` to `true` in the `logging` section of the configuration will display package build in the console. _This will prevent that output from going to the loki or file logger_, but can be helpful for bootstrapping and debugging.

```yaml
logging:
  live_build_output: true
```

## Developers

Your overlay can be a local directory if you are testing `template` configurations and don't want to commit every individual change:

```yaml
  overlays:
    CUSTOM:
      # git: https://gitlab.com/someuser/CUSTOM
      dir: /home/someuser/CUSTOM
```

`voidup` is built using [Poetry](https://python-poetry.org/). Void Linux has a package for building Poetry based projects, but it doesn't have a package for Poetry itself. Eventually I might get around to creating a Poetry package, but until then, developers will need to set up a virtual environment.

```shell
git clone https://gitlab.com/djsumdog/voidup
cd voidup
python -m venv venv
source venv/bin/active
pip install poetry
poetry install
poetry run python -m voidup --help
```

Instead of using the `/var/lib/voidup` locations, you can use the `--config` flag to specify an alternative configuration file. You can specify locations for all the build directories within your home directory:

```yaml
...
user: someuser
...
repositories:
  build:
    lib_dir: /home/someuser/voidup-dev
    repos_dir: /home/someuser/voidup-dev/repos
    work_dir: /home/someuser/voidup-dev/overlayfs/work
    upper_dir: /home/someuser/voidup-dev/overlayfs/upper
    merged_dir: /home/someuser/voidup-dev/overlayfs/merged
...
```

When developing, it's recommended to use the `--verbose` command for additional output. You will also need to run most of the commands as root (and therefore, activate the virtual environment as the root user), because only root is allowed to handle _overlayfs_ filesystems. `voidup` will drop to the user you specify on commands which do not require root privileges.

The local version will always be `0.0.0`. Before a release is made, the [poetry-git-version-plugin](https://gitlab.com/rocshers/python/poetry-git-version-plugin) is used to set the version in the `pyproject.toml` based off of git tags. Using the 1.x series of the plugin is necessary [due to an issue](https://gitlab.com/rocshers/python/poetry-git-version-plugin/-/issues/9). I do not commit the version back to git. It's only used within my deployment process to build the Python and xbps packages. 

```shell
source venv/bin/activate
pip install "poetry-git-version-plugin<2"
poetry set-git-version
poetry build
```

You can run tests using `poetry -m pytest -v` and code linting is checked via `flake8 --ignore=E501,W503 voidup tests` and `pycodestyle --ignore=E501,W504 voidup tests`.

## Known Issues and Roadmap

* The YAML configuration schema is not standard yet. It may change and break.
* ~~Multiple targets (different architectures or glibc vs muslc) are not supported yet. They are coming soon.~~
* _loki-grafana_ logging is not asynchronous. It will block and wait if your logging sever is down. I will implement queues in a future release.
* To build a single package, you'll have to comment the other packages out for now. I'll implement a command line switch for single packages.
