41 Commits

Author SHA1 Message Date
mdaleo404 29315de2ad Merge pull request 'Update pytest to 9.0.3' (#9) from pytest-update into main
Security Scan / security-scan (push) Successful in 1m25s
Reviewed-on: #9
2026-04-14 10:14:04 +00:00
mdaleo404 d7feb9c57c Update pytest to 9.0.3, filedust version bump
Lint & Security / precommit-and-security (pull_request) Successful in 1m58s
2026-04-14 11:11:09 +01:00
mdaleo404 dfe082389d Merge pull request 'Poetry update' (#8) from poetry-update-2.3.3 into main
Security Scan / security-scan (push) Failing after 1m22s
Reviewed-on: #8
2026-04-03 07:42:19 +00:00
mdaleo404 7a8d78ee61 Version bump
Lint & Security / precommit-and-security (pull_request) Successful in 1m1s
2026-04-03 08:40:52 +01:00
mdaleo404 ddc7286dbe Update Pygments
Lint & Security / precommit-and-security (pull_request) Successful in 1m1s
2026-04-03 08:38:36 +01:00
mdaleo404 b2156eb106 Poetry update
Lint & Security / precommit-and-security (pull_request) Failing after 1m1s
2026-04-03 08:30:39 +01:00
mdaleo404 94c13c44e4 Fix repo name in security-scan workflow
Security Scan / security-scan (push) Successful in 1m20s
2026-03-25 16:37:08 +00:00
mdaleo404 fcc9d19ae4 Exclude unfixed vulnerabilities from security workflow results 2026-03-25 16:34:37 +00:00
mdaleo404 8f80aba914 Switch Trivy scan to Syft and Grype 2026-03-25 16:13:49 +00:00
mdaleo404 32a009c85a Disable trivy scan workflow 2026-03-23 08:02:35 +00:00
mdaleo404 1b9d4cb903 Update pre-commit hooks version
Trivy Scan / security-scan (push) Successful in 27s
2026-03-21 07:25:47 +00:00
mdaleo404 d7063266ae Ping Trivy docker image to 0.69.3@sha256:bcc376de8d77cfe086a917230e818dc9f8528e3c852f7b1aff648949b6258d1c 2026-03-21 07:05:58 +00:00
mdaleo404 c090320e47 Update filelock and virtualenv
Trivy Scan / security-scan (push) Successful in 26s
2026-01-15 17:05:08 +00:00
mdaleo404 c7a07f8327 Add trivy-scan workflow 2026-01-15 16:50:25 +00:00
mdaleo404 31f91fcd28 Merge pull request 'Make pip-audit run inside Poetry' (#7) from pip_audit_tweak into main
Reviewed-on: #7
2025-12-25 10:30:06 +00:00
mdaleo404 5268e5834b Make pip-audit run inside Poetry
Lint & Security / precommit-and-security (pull_request) Successful in 59s
2025-12-25 10:28:46 +00:00
mdaleo404 532cc68fb3 Add logo file, update README 2025-12-21 08:44:09 +00:00
mdaleo404 e36228c308 Merge remote-tracking branch 'refs/remotes/origin/main' 2025-12-15 15:45:42 +00:00
mdaleo404 c016a45b82 Version bump 0.4.0 2025-12-15 15:43:48 +00:00
mdaleo404 860e8a668f Merge pull request 'Add globstar filtering' (#6) from add_globstar into main
Reviewed-on: #6
2025-12-15 15:42:15 +00:00
mdaleo404 5e9a55dcc6 Add globstar filtering
Lint & Security / precommit-and-security (pull_request) Successful in 1m28s
2025-12-15 15:39:14 +00:00
mdaleo404 7daa2175e8 Fix repository's URL to point ad Gitea 2025-12-14 16:42:43 +00:00
mdaleo404 67a6af2ddb Fix installation instructions 2025-12-09 16:11:49 +00:00
mdaleo404 25ea1fec63 Merge pull request 'Update README and pyproject.toml' (#5) from update_filedust_20251209 into main
Reviewed-on: #5
2025-12-09 15:27:51 +00:00
mdaleo404 dc66700f1e Edit badges, update installation instructions, swap github.com entries to git.sysmd.uk
Lint & Security / precommit-and-security (pull_request) Successful in 48s
2025-12-09 15:26:16 +00:00
mdaleo404 1eb082fc52 Merge pull request 'Rename .github folder to .gitea' (#4) from rename_github_folder into main
Reviewed-on: #4
2025-12-09 13:10:37 +00:00
mdaleo404 c2f52b8049 Use pre-commit directly instead of action
Lint & Security / precommit-and-security (pull_request) Successful in 1m3s
2025-12-09 13:08:10 +00:00
mdaleo404 6ebef8e058 Rename .github folder to .gitea
Lint & Security / precommit-and-security (pull_request) Has been cancelled
2025-12-09 12:56:06 +00:00
Marco D'Aleo fca4c8defc Merge pull request #3 from guardutils/relax_dependencies
Change dependencies constraints
2025-11-29 17:02:59 +00:00
mdaleo404 6cdfd2fc44 Change dependencies constraints, fix 'Looking for junk' print statement location 2025-11-29 17:01:22 +00:00
mdaleo404 6c1d2dc430 Update badges URLs 2025-11-29 16:40:13 +00:00
Marco D'Aleo fa8a194ccb Merge pull request #2 from guardutils/update_filedust_20251129
Improve sefety and add config file
- Add .cache and build to the skip dir list, make filedust run ONLY in the user home directory
- Major rewrite of junk.py, adding user config file for custom rules, don't treat broken symlink as junk
- Add filedust config file, update README, version bump
2025-11-29 10:52:40 +00:00
mdaleo404 677b14db26 Add filedust config file, update README, version bump 2025-11-29 10:23:05 +00:00
mdaleo404 35f5f2674a Major rewrite of junk.py, adding user config file for custom rules, don't treat broken symlink as junk 2025-11-29 10:02:45 +00:00
mdaleo404 c75a5246e3 Add .cache and build to the skip dir list, make filedust run ONLY in the user home directory 2025-11-29 08:29:14 +00:00
Marco D'Aleo 7f2b23b41b Merge pull request #1 from guardutils/update_filedust_20251127
Switch ownership from mdaleo404 to guardutils in README and pyproject
2025-11-27 17:44:01 +00:00
mdaleo404 ae281624da Trim trailing whitespaces in .gitignore 2025-11-27 17:42:49 +00:00
mdaleo404 1bebbcfa42 Switch ownership from mdaleo404 to guardutils in README and pyproject 2025-11-27 17:42:37 +00:00
mdaleo404 4e8171da84 Fix dependencies, add tab completion with argcomplete, update README 2025-11-26 12:59:38 +00:00
Marco D'Aleo dce1b271ce Update README.md 2025-11-23 21:09:26 +00:00
Marco D'Aleo f62a440890 Update README.md 2025-11-23 20:59:13 +00:00
12 changed files with 699 additions and 61 deletions
+23
View File
@@ -0,0 +1,23 @@
# filedust configuration file
# Place at: ~/.filedust.conf
#
# Use this file to customize cleanup behavior.
#
# Patterns are matched against paths relative to $HOME
# Supports:
# * = one path segment
# ** = zero or more path segments (recursive)
# Matching is case-sensitive
[exclude]
# Add directories or patterns you want filedust to ignore.
# Examples:
# Projects/important/*
[include]
# Add directories or patterns you want filedust to remove.
# Examples:
# node_modules
# dist
# *.tmp
# *~
@@ -20,10 +20,17 @@ jobs:
run: pip install pre-commit
- name: Run pre-commit hooks
uses: pre-commit/action@v3.0.1
run: pre-commit run --all-files --color always
- name: Install Poetry
run: |
pip install poetry
poetry self add poetry-plugin-export
- name: Install pip-audit
run: pip install pip-audit
- name: Run pip-audit
run: pip-audit
- name: Audit dependencies (Poetry lockfile)
run: |
poetry export -f requirements.txt --without-hashes \
| pip-audit -r /dev/stdin
+188
View File
@@ -0,0 +1,188 @@
name: Security Scan
on:
schedule:
- cron: 27 8 * * *
workflow_dispatch:
jobs:
security-scan:
runs-on: running-man
env:
TARGET_DIR: .
COSIGN_VERSION: v3.0.5
SYFT_VERSION: v1.42.3
GRYPE_VERSION: v0.110.0
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Install Cosign (bootstrap)
run: |
set -euo pipefail
FILE="cosign-linux-amd64"
curl -fLO https://github.com/sigstore/cosign/releases/download/${COSIGN_VERSION}/${FILE}
chmod +x ${FILE}
mv ${FILE} /usr/local/bin/cosign
cosign version
- name: Install Syft (verified)
run: |
set -euo pipefail
VERSION_NO_V="${SYFT_VERSION#v}"
FILE="syft_${VERSION_NO_V}_linux_amd64.tar.gz"
BASE_URL="https://github.com/anchore/syft/releases/download/${SYFT_VERSION}"
curl -fLO ${BASE_URL}/${FILE}
curl -fLO ${BASE_URL}/syft_${VERSION_NO_V}_checksums.txt
curl -fLO ${BASE_URL}/syft_${VERSION_NO_V}_checksums.txt.sig
curl -fLO ${BASE_URL}/syft_${VERSION_NO_V}_checksums.txt.pem
cosign verify-blob \
--signature syft_${VERSION_NO_V}_checksums.txt.sig \
--certificate syft_${VERSION_NO_V}_checksums.txt.pem \
--certificate-identity-regexp "https://github.com/anchore/syft" \
--certificate-oidc-issuer "https://token.actions.githubusercontent.com" \
syft_${VERSION_NO_V}_checksums.txt
CHECKSUM_LINE=$(grep " ${FILE}$" syft_${VERSION_NO_V}_checksums.txt)
if [ -z "$CHECKSUM_LINE" ]; then
echo "Missing checksum entry for ${FILE}"
exit 1
fi
echo "$CHECKSUM_LINE" | sha256sum -c -
tar -xzf ${FILE}
mv syft /usr/local/bin/
syft version
- name: Install Grype (verified)
run: |
set -euo pipefail
VERSION_NO_V="${GRYPE_VERSION#v}"
FILE="grype_${VERSION_NO_V}_linux_amd64.tar.gz"
BASE_URL="https://github.com/anchore/grype/releases/download/${GRYPE_VERSION}"
curl -fLO ${BASE_URL}/${FILE}
curl -fLO ${BASE_URL}/grype_${VERSION_NO_V}_checksums.txt
curl -fLO ${BASE_URL}/grype_${VERSION_NO_V}_checksums.txt.sig
curl -fLO ${BASE_URL}/grype_${VERSION_NO_V}_checksums.txt.pem
cosign verify-blob \
--signature grype_${VERSION_NO_V}_checksums.txt.sig \
--certificate grype_${VERSION_NO_V}_checksums.txt.pem \
--certificate-identity-regexp "https://github.com/anchore/grype" \
--certificate-oidc-issuer "https://token.actions.githubusercontent.com" \
grype_${VERSION_NO_V}_checksums.txt
CHECKSUM_LINE=$(grep " ${FILE}$" grype_${VERSION_NO_V}_checksums.txt)
if [ -z "$CHECKSUM_LINE" ]; then
echo "Missing checksum entry for ${FILE}"
exit 1
fi
echo "$CHECKSUM_LINE" | sha256sum -c -
tar -xzf ${FILE}
mv grype /usr/local/bin/
grype version
- name: Generate SBOM
working-directory: ${{ env.TARGET_DIR }}
run: |
syft dir:. -o json > sbom.json
- name: Show SBOM contents
working-directory: ${{ env.TARGET_DIR }}
run: |
echo "Packages discovered by Syft:"
jq -r '.artifacts[] | "\(.name)@\(.version) [\(.type)]"' sbom.json | sort
- name: Run Grype scan (JSON)
id: audit
continue-on-error: true
working-directory: ${{ env.TARGET_DIR }}
run: |
grype sbom:sbom.json -o json > grype.json
echo "Vulnerabilities (fixable only):"
jq -r '
.matches[]
| select((.vulnerability.fix.versions | length) > 0)
| "\(.artifact.name)@\(.artifact.version) -> \(.vulnerability.id) [\(.vulnerability.severity)] | fixed: \(.vulnerability.fix.versions[0])"
' grype.json
# Fail only on fixable MEDIUM/HIGH/CRITICAL
jq -e '
[
.matches[]?
| select(
(
.vulnerability.severity == "Medium" or
.vulnerability.severity == "High" or
.vulnerability.severity == "Critical"
)
and
(
(.vulnerability.fix.versions | length) > 0
)
)
]
| length == 0
' grype.json
- name: Show full Grype table
working-directory: ${{ env.TARGET_DIR }}
run: |
echo "Full Grype report:"
grype sbom:sbom.json -o table
- name: Notify Node-RED on vulnerabilities
if: steps.audit.outcome == 'failure'
working-directory: ${{ env.TARGET_DIR }}
run: |
jq '
{
repo: "guardutils/filedust",
summary: (
"Total: " +
(
[
.matches[]
| select((.vulnerability.fix.versions | length) > 0)
] | length | tostring
)
),
vulnerabilities: [
.matches[]
| select((.vulnerability.fix.versions | length) > 0)
| {
library: .artifact.name,
cve: .vulnerability.id,
severity: .vulnerability.severity,
installed: .artifact.version,
fixed: (.vulnerability.fix.versions[0]),
title: .vulnerability.description,
url: .vulnerability.dataSource
}
]
}
' grype.json \
| curl -s -X POST https://nodered.sysmd.uk/vulns-alert \
-H "Content-Type: application/json" \
--data-binary @-
- name: Fail workflow if vulnerabilities found
if: steps.audit.outcome == 'failure'
run: exit 1
+2 -2
View File
@@ -182,9 +182,9 @@ cython_debug/
.abstra/
# Visual Studio Code
# Visual Studio Code specific template is maintained in a separate VisualStudioCode.gitignore
# Visual Studio Code specific template is maintained in a separate VisualStudioCode.gitignore
# that can be found at https://github.com/github/gitignore/blob/main/Global/VisualStudioCode.gitignore
# and can be added to the global gitignore or merged into this file. However, if you prefer,
# and can be added to the global gitignore or merged into this file. However, if you prefer,
# you could uncomment the following to ignore the entire vscode folder
# .vscode/
+3 -3
View File
@@ -1,19 +1,19 @@
repos:
- repo: https://github.com/PyCQA/bandit
rev: 1.7.9
rev: 1.9.4
hooks:
- id: bandit
files: ^src/mirro/
args: ["-lll", "-iii", "-s", "B110,B112"]
- repo: https://github.com/psf/black-pre-commit-mirror
rev: 25.11.0
rev: 26.3.1
hooks:
- id: black
language_version: python3.13
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.4.0
rev: v6.0.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
+109 -7
View File
@@ -1,12 +1,13 @@
![License](https://img.shields.io/github/license/mdaleo404/filedust)
[![Language](https://img.shields.io/github/languages/top/mdaleo404/filedust.svg)](https://github.com/mdaleo404/filedust/)
![GitHub Release](https://img.shields.io/github/v/release/mdaleo404/filedust?display_name=release&logo=github)
![PyPI - Version](https://img.shields.io/pypi/v/filedust?logo=pypi)
[![Build Status](https://img.shields.io/github/actions/workflow/status/mdaleo404/filedust/.github/workflows/lint-and-security.yml)](https://github.com/mdaleo404/filedust/actions)
[![PyPI downloads](https://img.shields.io/pypi/dm/filedust.svg)](https://pypi.org/project/filedust/)
[![Licence](https://img.shields.io/badge/GPL--3.0-orange?label=Licence)](https://git.sysmd.uk/guardutils/filedust/src/branch/main/LICENCE)
[![Gitea Release](https://img.shields.io/gitea/v/release/guardutils/filedust?gitea_url=https%3A%2F%2Fgit.sysmd.uk%2F&style=flat&color=orange&logo=gitea)](https://git.sysmd.uk/guardutils/filedust/releases)
[![pre-commit](https://img.shields.io/badge/pre--commit-enabled-blue?logo=pre-commit&style=flat)](https://git.sysmd.uk/guardutils/filedust/src/branch/main/.pre-commit-config.yaml)
# filedust
<div align="center">
<img src="filedust.png" alt="filedust logo" width="256" />
</div>
**filedust** is a small, fast, and safe command-line tool that scans your filesystem for obvious junk — things like Python __pycache__ folders, build artifacts, editor backup files, and leftover temporary files — and cleans them up.
Think of it as “`autoremove` for files.”
@@ -35,4 +36,105 @@ One interactive prompt at the end of the run (unless -y is used).
Shows how much disk space can be freed.
### Safe by design
Never touches dotfiles, configs, project files, or anything important.
* It ONLY runs within user's `$HOME`
* Put user in control by reading `~/.filedust.conf`
* Never touches dotfiles, configs, project files, or anything important unless you want.
## Installation
### From GuardUtils package repo
This is the preferred method of installation.
### Debian/Ubuntu
#### 1) Import the GPG key
```bash
sudo mkdir -p /usr/share/keyrings
curl -fsSL https://repo.sysmd.uk/guardutils/guardutils.gpg | sudo gpg --dearmor -o /usr/share/keyrings/guardutils.gpg
```
The GPG fingerprint is `0032C71FA6A11EF9567D4434C5C06BD4603C28B1`.
#### 2) Add the APT source
```bash
echo "deb [arch=amd64 signed-by=/usr/share/keyrings/guardutils.gpg] https://repo.sysmd.uk/guardutils/debian stable main" | sudo tee /etc/apt/sources.list.d/guardutils.list
```
#### 3) Update and install
```
sudo apt update
sudo apt install filedust
```
### Fedora/RHEL
#### 1) Import the GPG key
```
sudo rpm --import https://repo.sysmd.uk/guardutils/guardutils.gpg
```
#### 2) Add the repository configuration
```
sudo tee /etc/yum.repos.d/guardutils.repo > /dev/null << 'EOF'
[guardutils]
name=GuardUtils Repository
baseurl=https://repo.sysmd.uk/guardutils/rpm/$basearch
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://repo.sysmd.uk/guardutils/guardutils.gpg
EOF
```
#### 4) Update and install
```
sudo dnf upgrade --refresh
sudo dnf install filedust
```
### From PyPI
```
pip install filedust
```
### From this repository
```
git clone https://git.sysmd.uk/guardutils/filedust.git
cd filedust/
poetry install
```
### Custom config
You can download the example and add your custom rule
```
wget -O ~/.filedust.conf https://git.sysmd.uk/guardutils/filedust/raw/branch/main/.filedust.conf.example
```
### TAB completion
Add this to your `.bashrc`
```
eval "$(register-python-argcomplete filedust)"
```
And then
```
source ~/.bashrc
```
## pre-commit
This project uses [**pre-commit**](https://pre-commit.com/) to run automatic formatting and security checks before each commit (Black, Bandit, and various safety checks).
To enable it:
```
poetry install
poetry run pre-commit install
```
This ensures consistent formatting, catches common issues early, and keeps the codebase clean.
BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 35 KiB

Generated
+114 -19
View File
@@ -1,4 +1,19 @@
# This file is automatically @generated by Poetry 1.8.4 and should not be changed by hand.
# This file is automatically @generated by Poetry 2.3.3 and should not be changed by hand.
[[package]]
name = "argcomplete"
version = "3.6.3"
description = "Bash tab completion for argparse"
optional = false
python-versions = ">=3.8"
groups = ["main"]
files = [
{file = "argcomplete-3.6.3-py3-none-any.whl", hash = "sha256:f5007b3a600ccac5d25bbce33089211dfd49eab4a7718da3f10e3082525a92ce"},
{file = "argcomplete-3.6.3.tar.gz", hash = "sha256:62e8ed4fd6a45864acc8235409461b72c9a28ee785a2011cc5eb78318786c89c"},
]
[package.extras]
test = ["coverage", "mypy", "pexpect", "ruff", "wheel"]
[[package]]
name = "cfgv"
@@ -6,6 +21,7 @@ version = "3.5.0"
description = "Validate configuration and produce human readable error messages."
optional = false
python-versions = ">=3.10"
groups = ["dev"]
files = [
{file = "cfgv-3.5.0-py2.py3-none-any.whl", hash = "sha256:a8dc6b26ad22ff227d2634a65cb388215ce6cc96bbcc5cfde7641ae87e8dacc0"},
{file = "cfgv-3.5.0.tar.gz", hash = "sha256:d5b1034354820651caa73ede66a6294d6e95c1b00acc5e9b098e917404669132"},
@@ -17,6 +33,8 @@ version = "0.4.6"
description = "Cross-platform colored terminal text."
optional = false
python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,!=3.5.*,!=3.6.*,>=2.7"
groups = ["dev"]
markers = "sys_platform == \"win32\""
files = [
{file = "colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6"},
{file = "colorama-0.4.6.tar.gz", hash = "sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44"},
@@ -28,6 +46,7 @@ version = "7.12.0"
description = "Code coverage measurement for Python"
optional = false
python-versions = ">=3.10"
groups = ["dev"]
files = [
{file = "coverage-7.12.0-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:32b75c2ba3f324ee37af3ccee5b30458038c50b349ad9b88cee85096132a575b"},
{file = "coverage-7.12.0-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:cb2a1b6ab9fe833714a483a915de350abc624a37149649297624c8d57add089c"},
@@ -127,7 +146,7 @@ files = [
tomli = {version = "*", optional = true, markers = "python_full_version <= \"3.11.0a6\" and extra == \"toml\""}
[package.extras]
toml = ["tomli"]
toml = ["tomli ; python_full_version <= \"3.11.0a6\""]
[[package]]
name = "distlib"
@@ -135,6 +154,7 @@ version = "0.4.0"
description = "Distribution utilities"
optional = false
python-versions = "*"
groups = ["dev"]
files = [
{file = "distlib-0.4.0-py2.py3-none-any.whl", hash = "sha256:9659f7d87e46584a30b5780e43ac7a2143098441670ff0a49d5f9034c54a6c16"},
{file = "distlib-0.4.0.tar.gz", hash = "sha256:feec40075be03a04501a973d81f633735b4b69f98b05450592310c0f401a4e0d"},
@@ -146,6 +166,8 @@ version = "1.3.1"
description = "Backport of PEP 654 (exception groups)"
optional = false
python-versions = ">=3.7"
groups = ["dev"]
markers = "python_version == \"3.10\""
files = [
{file = "exceptiongroup-1.3.1-py3-none-any.whl", hash = "sha256:a7a39a3bd276781e98394987d3a5701d0c4edffb633bb7a5144577f82c773598"},
{file = "exceptiongroup-1.3.1.tar.gz", hash = "sha256:8b412432c6055b0b7d14c310000ae93352ed6754f70fa8f7c34141f91c4e3219"},
@@ -159,13 +181,14 @@ test = ["pytest (>=6)"]
[[package]]
name = "filelock"
version = "3.20.0"
version = "3.20.3"
description = "A platform independent file lock."
optional = false
python-versions = ">=3.10"
groups = ["dev"]
files = [
{file = "filelock-3.20.0-py3-none-any.whl", hash = "sha256:339b4732ffda5cd79b13f4e2711a31b0365ce445d95d243bb996273d072546a2"},
{file = "filelock-3.20.0.tar.gz", hash = "sha256:711e943b4ec6be42e1d4e6690b48dc175c822967466bb31c0c293f34334c13f4"},
{file = "filelock-3.20.3-py3-none-any.whl", hash = "sha256:4b0dda527ee31078689fc205ec4f1c1bf7d56cf88b6dc9426c4f230e46c2dce1"},
{file = "filelock-3.20.3.tar.gz", hash = "sha256:18c57ee915c7ec61cff0ecf7f0f869936c7c30191bb0cf406f1341778d0834e1"},
]
[[package]]
@@ -174,6 +197,7 @@ version = "2.6.15"
description = "File identification library for Python"
optional = false
python-versions = ">=3.9"
groups = ["dev"]
files = [
{file = "identify-2.6.15-py2.py3-none-any.whl", hash = "sha256:1181ef7608e00704db228516541eb83a88a9f94433a8c80bb9b5bd54b1d81757"},
{file = "identify-2.6.15.tar.gz", hash = "sha256:e4f4864b96c6557ef2a1e1c951771838f4edc9df3a72ec7118b338801b11c7bf"},
@@ -188,17 +212,55 @@ version = "2.3.0"
description = "brain-dead simple config-ini parsing"
optional = false
python-versions = ">=3.10"
groups = ["dev"]
files = [
{file = "iniconfig-2.3.0-py3-none-any.whl", hash = "sha256:f631c04d2c48c52b84d0d0549c99ff3859c98df65b3101406327ecc7d53fbf12"},
{file = "iniconfig-2.3.0.tar.gz", hash = "sha256:c76315c77db068650d49c5b56314774a7804df16fee4402c1f19d6d15d8c4730"},
]
[[package]]
name = "markdown-it-py"
version = "4.0.0"
description = "Python port of markdown-it. Markdown parsing, done right!"
optional = false
python-versions = ">=3.10"
groups = ["main"]
files = [
{file = "markdown_it_py-4.0.0-py3-none-any.whl", hash = "sha256:87327c59b172c5011896038353a81343b6754500a08cd7a4973bb48c6d578147"},
{file = "markdown_it_py-4.0.0.tar.gz", hash = "sha256:cb0a2b4aa34f932c007117b194e945bd74e0ec24133ceb5bac59009cda1cb9f3"},
]
[package.dependencies]
mdurl = ">=0.1,<1.0"
[package.extras]
benchmarking = ["psutil", "pytest", "pytest-benchmark"]
compare = ["commonmark (>=0.9,<1.0)", "markdown (>=3.4,<4.0)", "markdown-it-pyrs", "mistletoe (>=1.0,<2.0)", "mistune (>=3.0,<4.0)", "panflute (>=2.3,<3.0)"]
linkify = ["linkify-it-py (>=1,<3)"]
plugins = ["mdit-py-plugins (>=0.5.0)"]
profiling = ["gprof2dot"]
rtd = ["ipykernel", "jupyter_sphinx", "mdit-py-plugins (>=0.5.0)", "myst-parser", "pyyaml", "sphinx", "sphinx-book-theme (>=1.0,<2.0)", "sphinx-copybutton", "sphinx-design"]
testing = ["coverage", "pytest", "pytest-cov", "pytest-regressions", "requests"]
[[package]]
name = "mdurl"
version = "0.1.2"
description = "Markdown URL utilities"
optional = false
python-versions = ">=3.7"
groups = ["main"]
files = [
{file = "mdurl-0.1.2-py3-none-any.whl", hash = "sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8"},
{file = "mdurl-0.1.2.tar.gz", hash = "sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba"},
]
[[package]]
name = "nodeenv"
version = "1.9.1"
description = "Node.js virtual environment builder"
optional = false
python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,!=3.5.*,!=3.6.*,>=2.7"
groups = ["dev"]
files = [
{file = "nodeenv-1.9.1-py2.py3-none-any.whl", hash = "sha256:ba11c9782d29c27c70ffbdda2d7415098754709be8a7056d79a737cd901155c9"},
{file = "nodeenv-1.9.1.tar.gz", hash = "sha256:6ec12890a2dab7946721edbfbcd91f3319c6ccc9aec47be7c7e6b7011ee6645f"},
@@ -210,6 +272,7 @@ version = "25.0"
description = "Core utilities for Python packages"
optional = false
python-versions = ">=3.8"
groups = ["dev"]
files = [
{file = "packaging-25.0-py3-none-any.whl", hash = "sha256:29572ef2b1f17581046b3a2227d5c611fb25ec70ca1ba8554b24b0e69331a484"},
{file = "packaging-25.0.tar.gz", hash = "sha256:d443872c98d677bf60f6a1f2f8c1cb748e8fe762d2bf9d3148b5599295b0fc4f"},
@@ -221,6 +284,7 @@ version = "4.5.0"
description = "A small Python package for determining appropriate platform-specific dirs, e.g. a `user data dir`."
optional = false
python-versions = ">=3.10"
groups = ["dev"]
files = [
{file = "platformdirs-4.5.0-py3-none-any.whl", hash = "sha256:e578a81bb873cbb89a41fcc904c7ef523cc18284b7e3b3ccf06aca1403b7ebd3"},
{file = "platformdirs-4.5.0.tar.gz", hash = "sha256:70ddccdd7c99fc5942e9fc25636a8b34d04c24b335100223152c2803e4063312"},
@@ -237,6 +301,7 @@ version = "1.6.0"
description = "plugin and hook calling mechanisms for python"
optional = false
python-versions = ">=3.9"
groups = ["dev"]
files = [
{file = "pluggy-1.6.0-py3-none-any.whl", hash = "sha256:e920276dd6813095e9377c0bc5566d94c932c33b27a3e3945d8389c374dd4746"},
{file = "pluggy-1.6.0.tar.gz", hash = "sha256:7dcc130b76258d33b90f61b658791dede3486c3e6bfb003ee5c9bfb396dd22f3"},
@@ -252,6 +317,7 @@ version = "3.8.0"
description = "A framework for managing and maintaining multi-language pre-commit hooks."
optional = false
python-versions = ">=3.9"
groups = ["dev"]
files = [
{file = "pre_commit-3.8.0-py2.py3-none-any.whl", hash = "sha256:9a90a53bf82fdd8778d58085faf8d83df56e40dfe18f45b19446e26bf1b3a63f"},
{file = "pre_commit-3.8.0.tar.gz", hash = "sha256:8bb6494d4a20423842e198980c9ecf9f96607a07ea29549e180eef9ae80fe7af"},
@@ -266,13 +332,14 @@ virtualenv = ">=20.10.0"
[[package]]
name = "pygments"
version = "2.19.2"
version = "2.20.0"
description = "Pygments is a syntax highlighting package written in Python."
optional = false
python-versions = ">=3.8"
python-versions = ">=3.9"
groups = ["main", "dev"]
files = [
{file = "pygments-2.19.2-py3-none-any.whl", hash = "sha256:86540386c03d588bb81d44bc3928634ff26449851e99741617ecb9037ee5ec0b"},
{file = "pygments-2.19.2.tar.gz", hash = "sha256:636cb2477cec7f8952536970bc533bc43743542f70392ae026374600add5b887"},
{file = "pygments-2.20.0-py3-none-any.whl", hash = "sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176"},
{file = "pygments-2.20.0.tar.gz", hash = "sha256:6757cd03768053ff99f3039c1a36d6c0aa0b263438fcab17520b30a303a82b5f"},
]
[package.extras]
@@ -280,13 +347,14 @@ windows-terminal = ["colorama (>=0.4.6)"]
[[package]]
name = "pytest"
version = "9.0.1"
version = "9.0.3"
description = "pytest: simple powerful testing with Python"
optional = false
python-versions = ">=3.10"
groups = ["dev"]
files = [
{file = "pytest-9.0.1-py3-none-any.whl", hash = "sha256:67be0030d194df2dfa7b556f2e56fb3c3315bd5c8822c6951162b92b32ce7dad"},
{file = "pytest-9.0.1.tar.gz", hash = "sha256:3e9c069ea73583e255c3b21cf46b8d3c56f6e3a1a8f6da94ccb0fcf57b9d73c8"},
{file = "pytest-9.0.3-py3-none-any.whl", hash = "sha256:2c5efc453d45394fdd706ade797c0a81091eccd1d6e4bccfcd476e2b8e0ab5d9"},
{file = "pytest-9.0.3.tar.gz", hash = "sha256:b86ada508af81d19edeb213c681b1d48246c1a91d304c6c81a427674c17eb91c"},
]
[package.dependencies]
@@ -307,6 +375,7 @@ version = "7.0.0"
description = "Pytest plugin for measuring coverage."
optional = false
python-versions = ">=3.9"
groups = ["dev"]
files = [
{file = "pytest_cov-7.0.0-py3-none-any.whl", hash = "sha256:3b8e9558b16cc1479da72058bdecf8073661c7f57f7d3c5f22a1c23507f2d861"},
{file = "pytest_cov-7.0.0.tar.gz", hash = "sha256:33c97eda2e049a0c5298e91f519302a1334c26ac65c1a483d6206fd458361af1"},
@@ -326,6 +395,7 @@ version = "6.0.3"
description = "YAML parser and emitter for Python"
optional = false
python-versions = ">=3.8"
groups = ["dev"]
files = [
{file = "PyYAML-6.0.3-cp38-cp38-macosx_10_13_x86_64.whl", hash = "sha256:c2514fceb77bc5e7a2f7adfaa1feb2fb311607c9cb518dbc378688ec73d8292f"},
{file = "PyYAML-6.0.3-cp38-cp38-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:9c57bb8c96f6d1808c030b1687b9b5fb476abaa47f0db9c0101f5e9f394e97f4"},
@@ -402,12 +472,34 @@ files = [
{file = "pyyaml-6.0.3.tar.gz", hash = "sha256:d76623373421df22fb4cf8817020cbb7ef15c725b9d5e45f17e189bfc384190f"},
]
[[package]]
name = "rich"
version = "13.9.4"
description = "Render rich text, tables, progress bars, syntax highlighting, markdown and more to the terminal"
optional = false
python-versions = ">=3.8.0"
groups = ["main"]
files = [
{file = "rich-13.9.4-py3-none-any.whl", hash = "sha256:6049d5e6ec054bf2779ab3358186963bac2ea89175919d699e378b99738c2a90"},
{file = "rich-13.9.4.tar.gz", hash = "sha256:439594978a49a09530cff7ebc4b5c7103ef57baf48d5ea3184f21d9a2befa098"},
]
[package.dependencies]
markdown-it-py = ">=2.2.0"
pygments = ">=2.13.0,<3.0.0"
typing-extensions = {version = ">=4.0.0,<5.0", markers = "python_version < \"3.11\""}
[package.extras]
jupyter = ["ipywidgets (>=7.5.1,<9)"]
[[package]]
name = "tomli"
version = "2.3.0"
description = "A lil' TOML parser"
optional = false
python-versions = ">=3.8"
groups = ["dev"]
markers = "python_full_version <= \"3.11.0a6\""
files = [
{file = "tomli-2.3.0-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:88bd15eb972f3664f5ed4b57c1634a97153b4bac4479dcb6a495f41921eb7f45"},
{file = "tomli-2.3.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:883b1c0d6398a6a9d29b508c331fa56adbcdff647f6ace4dfca0f50e90dfd0ba"},
@@ -459,6 +551,8 @@ version = "4.15.0"
description = "Backported and Experimental Type Hints for Python 3.9+"
optional = false
python-versions = ">=3.9"
groups = ["main", "dev"]
markers = "python_version == \"3.10\""
files = [
{file = "typing_extensions-4.15.0-py3-none-any.whl", hash = "sha256:f0fa19c6845758ab08074a0cfa8b7aecb71c999ca73d62883bc25cc018c4e548"},
{file = "typing_extensions-4.15.0.tar.gz", hash = "sha256:0cea48d173cc12fa28ecabc3b837ea3cf6f38c6d1136f85cbaaf598984861466"},
@@ -466,26 +560,27 @@ files = [
[[package]]
name = "virtualenv"
version = "20.35.4"
version = "20.36.1"
description = "Virtual Python Environment builder"
optional = false
python-versions = ">=3.8"
groups = ["dev"]
files = [
{file = "virtualenv-20.35.4-py3-none-any.whl", hash = "sha256:c21c9cede36c9753eeade68ba7d523529f228a403463376cf821eaae2b650f1b"},
{file = "virtualenv-20.35.4.tar.gz", hash = "sha256:643d3914d73d3eeb0c552cbb12d7e82adf0e504dbf86a3182f8771a153a1971c"},
{file = "virtualenv-20.36.1-py3-none-any.whl", hash = "sha256:575a8d6b124ef88f6f51d56d656132389f961062a9177016a50e4f507bbcc19f"},
{file = "virtualenv-20.36.1.tar.gz", hash = "sha256:8befb5c81842c641f8ee658481e42641c68b5eab3521d8e092d18320902466ba"},
]
[package.dependencies]
distlib = ">=0.3.7,<1"
filelock = ">=3.12.2,<4"
filelock = {version = ">=3.20.1,<4", markers = "python_version >= \"3.10\""}
platformdirs = ">=3.9.1,<5"
typing-extensions = {version = ">=4.13.2", markers = "python_version < \"3.11\""}
[package.extras]
docs = ["furo (>=2023.7.26)", "proselint (>=0.13)", "sphinx (>=7.1.2,!=7.3)", "sphinx-argparse (>=0.4)", "sphinxcontrib-towncrier (>=0.2.1a0)", "towncrier (>=23.6)"]
test = ["covdefaults (>=2.3)", "coverage (>=7.2.7)", "coverage-enable-subprocess (>=1)", "flaky (>=3.7)", "packaging (>=23.1)", "pytest (>=7.4)", "pytest-env (>=0.8.2)", "pytest-freezer (>=0.4.8)", "pytest-mock (>=3.11.1)", "pytest-randomly (>=3.12)", "pytest-timeout (>=2.1)", "setuptools (>=68)", "time-machine (>=2.10)"]
test = ["covdefaults (>=2.3)", "coverage (>=7.2.7)", "coverage-enable-subprocess (>=1)", "flaky (>=3.7)", "packaging (>=23.1)", "pytest (>=7.4)", "pytest-env (>=0.8.2)", "pytest-freezer (>=0.4.8) ; platform_python_implementation == \"PyPy\" or platform_python_implementation == \"GraalVM\" or platform_python_implementation == \"CPython\" and sys_platform == \"win32\" and python_version >= \"3.13\"", "pytest-mock (>=3.11.1)", "pytest-randomly (>=3.12)", "pytest-timeout (>=2.1)", "setuptools (>=68)", "time-machine (>=2.10) ; platform_python_implementation == \"CPython\""]
[metadata]
lock-version = "2.0"
lock-version = "2.1"
python-versions = ">=3.10,<4.0"
content-hash = "98acd9fd57ec90c98a407b83122fd9c8ed432383e095a47d44e201bf187d3107"
content-hash = "45b6935eff03de6a71792d1a2ea6c4d5ffa5f2d55046647a47268e798c22bb51"
+6 -4
View File
@@ -1,21 +1,23 @@
[tool.poetry]
name = "filedust"
version = "0.1.0"
version = "0.4.3"
description = "Opinionated junk cleaner for dev machines (caches, build artifacts, editor backups)."
authors = ["Marco D'Aleo <marco@marcodaleo.com>"]
license = "GPL-3.0-or-later"
readme = "README.md"
homepage = "https://github.com/mdaleo404/filedust"
repository = "https://github.com/mdaleo404/filedust"
homepage = "https://git.sysmd.uk/guardutils/filedust"
repository = "https://git.sysmd.uk/guardutils/filedust"
packages = [{ include = "filedust", from = "src" }]
[tool.poetry.dependencies]
python = ">=3.10,<4.0"
rich = ">=12"
argcomplete = ">=2"
[tool.poetry.scripts]
filedust = "filedust.cli:main"
[tool.poetry.dev-dependencies]
[tool.poetry.group.dev.dependencies]
pytest = "^9.0.1"
pytest-cov = "^7.0.0"
pre-commit = "^3.8"
+24 -3
View File
@@ -2,6 +2,7 @@ from __future__ import annotations
import importlib.metadata
import argparse
import argcomplete
import shutil
from pathlib import Path
from typing import List
@@ -11,8 +12,7 @@ from rich.table import Table
from rich.prompt import Confirm
from rich import box
from .junk import Finding, iter_junk
from .junk import Finding, iter_junk, load_user_rules
console = Console()
@@ -96,6 +96,11 @@ def build_parser() -> argparse.ArgumentParser:
help="Delete without prompting for confirmation.",
)
try:
argcomplete.autocomplete(parser)
except ImportError:
pass
return parser
@@ -166,18 +171,34 @@ def main(argv: list[str] | None = None) -> int:
args = parser.parse_args(argv)
root = Path(args.path).expanduser()
home = Path.home().resolve()
root_resolved = root.resolve()
# Ensure root is inside the user's home directory
try:
root_resolved.relative_to(home)
except ValueError:
console.print(
f"[red]Error:[/] Refusing to operate outside the user's home directory.\n"
f"Requested: {root_resolved}\n"
f"Allowed: {home}"
)
return 1
if not root.exists():
console.print(f"[red]Error:[/] Path not found: {root}")
return 1
print("Looking for junk ...")
if root.resolve() == Path("/"):
console.print(
"[yellow]Running filedust on the entire filesystem (/). "
"This may take a while and may require sudo for deletions.[/]"
)
findings = list(iter_junk(root))
rules = load_user_rules()
findings = list(iter_junk(root, rules=rules))
total_size = compute_total_size(findings)
if not findings:
+220 -20
View File
@@ -1,12 +1,80 @@
from __future__ import annotations
import os
import configparser
from dataclasses import dataclass
from fnmatch import fnmatch
from pathlib import Path
from typing import Iterable, List
class UserRules:
def __init__(self):
self.include: list[str] = []
self.exclude: list[str] = []
def load_user_rules() -> UserRules:
rules = UserRules()
cfg_path = Path.home() / ".filedust.conf"
if cfg_path.exists():
parser = configparser.ConfigParser(allow_no_value=True)
parser.optionxform = str
parser.read(cfg_path)
if parser.has_section("include"):
rules.include = list(parser["include"].keys())
if parser.has_section("exclude"):
rules.exclude = list(parser["exclude"].keys())
return rules
def matches_any(patterns: list[str], relpath: Path) -> bool:
"""
True globstar matcher.
Rules:
- * matches exactly one path segment
- ** matches zero or more segments
- Patterns are relative to $HOME
"""
path_parts = relpath.parts
for pat in patterns:
pat = pat.strip("/")
pat_parts = tuple(pat.split("/"))
if _match_parts(pat_parts, path_parts):
return True
return False
def _match_parts(pat: tuple[str, ...], path: tuple[str, ...]) -> bool:
"""Recursive glob matcher with ** support."""
if not pat:
return not path
if pat[0] == "**":
# ** matches zero or more segments
return _match_parts(pat[1:], path) or (
bool(path) and _match_parts(pat, path[1:])
)
if not path:
return False
if fnmatch(path[0], pat[0]):
return _match_parts(pat[1:], path[1:])
return False
@dataclass
class Finding:
path: Path
@@ -23,7 +91,6 @@ JUNK_DIR_NAMES = {
".nox",
".tox",
".hypothesis",
".cache",
".gradle",
".parcel-cache",
".turbo",
@@ -31,7 +98,6 @@ JUNK_DIR_NAMES = {
".vite",
".sass-cache",
".sass-cache",
"build",
"dist",
}
@@ -53,6 +119,9 @@ JUNK_FILE_PATTERNS = [
# VCS / system dirs
SKIP_DIR_NAMES = {
".cache",
"build",
".gnupg",
".git",
".hg",
".svn",
@@ -62,6 +131,34 @@ SKIP_DIR_NAMES = {
}
HOME = Path.home().resolve()
def safe_exists(path: Path) -> bool | None:
"""Return True/False if the path exists, or None if permission denied."""
try:
return path.exists()
except Exception:
return None
def safe_resolve(path: Path, root: Path) -> Path | None:
"""
Resolve symlinks only if safe.
Return resolved path if it stays within root.
Return None if:
- resolution escapes the root
- resolution fails
- permission denied
"""
try:
resolved = path.resolve(strict=False) # NEVER strict
resolved.relative_to(root) # ensure containment
return resolved
except Exception:
return None
def is_junk_dir_name(name: str) -> bool:
return name in JUNK_DIR_NAMES
@@ -70,37 +167,140 @@ def is_junk_file_name(name: str) -> bool:
return any(fnmatch(name, pattern) for pattern in JUNK_FILE_PATTERNS)
def iter_junk(root: Path) -> Iterable[Finding]:
def iter_junk(root: Path, rules: UserRules | None = None) -> Iterable[Finding]:
"""
Walk the tree under `root` and yield junk candidates.
Safe, fast junk scanner:
- Never follows symlinks.
- Broken symlinks are not automatically junk — they follow normal rules.
- User include/exclude overrides all.
- Built-in junk rules applied only when safe.
- SKIP_DIR_NAMES protected unless user includes.
- Fully contained in $HOME.
- No crashes from PermissionError or unreadable paths.
"""
if rules is None:
rules = UserRules()
filedust:
- Skips known critical / config directories (SKIP_DIR_NAMES).
- Treats known "junk" directory names as removable as a whole.
- Treats known junk file patterns as removable.
"""
root = root.resolve()
root_str = str(root)
for dirpath, dirnames, filenames in os.walk(root):
for dirpath, dirnames, filenames in os.walk(root, followlinks=False):
dirpath_p = Path(dirpath)
# Prune dirs we never touch at all.
dirnames[:] = [d for d in dirnames if d not in SKIP_DIR_NAMES]
try:
rel_dir = dirpath_p.resolve().relative_to(HOME)
except ValueError:
# Should never happen due to earlier checks
continue
# Detect junk directories (and skip walking inside them).
# USER EXCLUDE → skip entire subtree
if matches_any(rules.exclude, rel_dir):
dirnames[:] = []
continue
pruned = []
# Handling dirs
for d in dirnames:
child = dirpath_p / d
try:
st = child.lstat()
except Exception:
continue # unreadable
is_symlink = (st.st_mode & 0o170000) == 0o120000
if is_symlink:
# If broken symlink dir treat as file later via filenames (skip descent)
continue
rel_child = rel_dir / d
# User exclude wins
if matches_any(rules.exclude, rel_child):
continue
# SKIP_DIR_NAMES unless user includes
if d in SKIP_DIR_NAMES and not matches_any(
rules.include, rel_child
):
continue
pruned.append(d)
dirnames[:] = pruned
# Detect JUNK dirs
i = 0
while i < len(dirnames):
name = dirnames[i]
if is_junk_dir_name(name):
junk_dir = dirpath_p / name
yield Finding(path=junk_dir, kind="dir", reason="junk_dir")
# Remove from walk so we don't descend into it.
rel_child = rel_dir / name
# User include directory
if matches_any(rules.include, rel_child):
yield Finding(dirpath_p / name, "dir", "user_include")
del dirnames[i]
continue
# Built-in safe junk dirs
if is_junk_dir_name(name):
yield Finding(dirpath_p / name, "dir", "junk_dir")
del dirnames[i]
continue
i += 1
# Now process files.
# Handling files (including symlinks)
for fname in filenames:
fpath = dirpath_p / fname
rel_file = rel_dir / fname
try:
st = fpath.lstat()
except Exception:
continue
is_symlink = (st.st_mode & 0o170000) == 0o120000
# Handling broken symlinks
if is_symlink:
exists = safe_exists(fpath)
# Permission denied → skip
if exists is None:
continue
# User exclude wins
if matches_any(rules.exclude, rel_file):
continue
# User include wins
if matches_any(rules.include, rel_file):
yield Finding(fpath, "file", "user_include")
continue
# Broken symlink?
if exists is False:
# DO NOT auto-delete — classify like regular file
# Only built-in junk patterns apply
if is_junk_file_name(fname):
yield Finding(fpath, "file", "broken_symlink")
continue
# Valid symlink — NEVER follow; only user-include counts
continue
# Regular files
# User exclude wins
if matches_any(rules.exclude, rel_file):
continue
# User include wins
if matches_any(rules.include, rel_file):
yield Finding(fpath, "file", "user_include")
continue
# Built-in junk patterns (safe ones)
if is_junk_file_name(fname):
fpath = dirpath_p / fname
yield Finding(path=fpath, kind="file", reason="junk_file")
yield Finding(fpath, "file", "junk_file")