Skip to content

Commit 71fca50

Browse files
authored
Merge pull request #2 from chainguard-dev/shellcheck
Add a hook to run shellcheck on all "runs" steps in melange pipelines
2 parents f6097c0 + 54a94fa commit 71fca50

9 files changed

Lines changed: 280 additions & 22 deletions

File tree

.github/workflows/lint.yaml

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
name: Lint
2+
3+
on:
4+
pull_request:
5+
branches: ['main']
6+
7+
permissions:
8+
contents: read
9+
10+
jobs:
11+
lint:
12+
name: Lint
13+
runs-on: ubuntu-latest
14+
15+
permissions:
16+
contents: read
17+
18+
steps:
19+
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # ratchet:actions/checkout@v4.2.2
20+
- uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b # v5.3.0
21+
- uses: pre-commit/action@2c7b3805fd2a0fd8c1884dcaebf91fc102a13ecd # v3.0.1

.pre-commit-config.yaml

Lines changed: 37 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -1,25 +1,6 @@
1-
# In general, this file (when sourced from https://github.com/chainguard-dev/pre-commit-hooks)
2-
# should be able to be used as-is in other repos such as wolfi-dev/os, enterprise-packages, and extra-packages,
3-
# and can be assumed to be the "source of truth" for our pre-commit rules.
4-
# See https://eng.inky.wtf/docs/technical/git/pre-commit for info on how to edit this file.
1+
# Update with `pre-commit autoupdate --freeze` which
2+
# pins all repos using commit hashes, not mutable references
53
repos:
6-
- repo: https://github.com/chainguard-dev/pre-commit-hooks
7-
rev: e4f3bba353cc583ce73f660dcf217e245fd681d3
8-
hooks:
9-
- id: check-for-epoch-bump
10-
- repo: https://github.com/chainguard-dev/yam
11-
rev: 498642e77997ba79709f43a7ee2c84b12b2145bb # v0.2.12
12-
hooks:
13-
- id: yam
14-
# TODO: Swap with CG repo: - repo: https://github.com/wolfi-dev/wolfictl
15-
- repo: https://github.com/amberarcadia/wolfictl
16-
rev: 85b1301c4d17fcd0c8f0ce455724941cae815d68
17-
hooks:
18-
- id: wolfictl-lint
19-
- repo: https://github.com/golangci/misspell
20-
rev: 528d713e620bdf4b41849db93cb489c4fef9f5c5 # v0.6.0
21-
hooks:
22-
- id: misspell
234
- repo: https://github.com/pre-commit/pre-commit-hooks
245
rev: cef0300fd0fc4d2a87a85fa2093c6b283ea36f4b # v5.0.0
256
hooks:
@@ -30,4 +11,38 @@ repos:
3011
- id: check-merge-conflict
3112
- id: check-symlinks
3213
- id: detect-private-key
33-
exclude: ^ruby-3\.0/0001-ruby-3\.0\.6-openssl-patch\.patch$
14+
- repo: https://github.com/astral-sh/ruff-pre-commit
15+
rev: 12753357c00c3fb8615100354c9fdc6ab80b044d # frozen: v0.11.10
16+
hooks:
17+
- id: ruff-check
18+
- id: ruff-format
19+
- repo: https://github.com/asottile/setup-cfg-fmt
20+
rev: 79cc0ae5abfa1ba092b5938cd811a6069710ad77 # frozen: v2.8.0
21+
hooks:
22+
- id: setup-cfg-fmt
23+
- repo: https://github.com/asottile/reorder-python-imports
24+
rev: fd0b4e1292716bcd12a396b86af1d1271aaaa62c # frozen: v3.14.0
25+
hooks:
26+
- id: reorder-python-imports
27+
args: [--py39-plus, --add-import, 'from __future__ import annotations']
28+
- repo: https://github.com/asottile/add-trailing-comma
29+
rev: d2e6adc1665e461a764e2f38edfa2ef61f41be20 # frozen: v3.1.0
30+
hooks:
31+
- id: add-trailing-comma
32+
- repo: https://github.com/asottile/pyupgrade
33+
rev: ce40a160603ab0e7d9c627ae33d7ef3906e2d2b2 # frozen: v3.19.1
34+
hooks:
35+
- id: pyupgrade
36+
args: [--py39-plus]
37+
- repo: https://github.com/hhatto/autopep8
38+
rev: 4046ad49e25b7fa1db275bf66b1b7d60600ac391 # frozen: v2.3.2
39+
hooks:
40+
- id: autopep8
41+
- repo: https://github.com/PyCQA/flake8
42+
rev: 4b5e89b4b108a6c1a000c591d334a99a80d34c7b # frozen: 7.2.0
43+
hooks:
44+
- id: flake8
45+
- repo: https://github.com/pre-commit/mirrors-mypy
46+
rev: f40886d54c729f533f864ed6ce584e920feb0af7 # frozen: v1.15.0
47+
hooks:
48+
- id: mypy

.pre-commit-hooks.yaml

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,3 +10,13 @@
1010
- manual
1111
types:
1212
- yaml
13+
- id: shellcheck-run-steps
14+
name: shellcheck run steps
15+
description: run shellcheck on each "run" step in a melange pipeline
16+
entry: shellcheck-run-steps
17+
language: python
18+
stages:
19+
- pre-commit
20+
- manual
21+
types:
22+
- yaml

example.pre-commit-config.yaml

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
# In general, this file (when sourced from https://github.com/chainguard-dev/pre-commit-hooks)
2+
# should be able to be used as-is in other repos such as wolfi-dev/os, enterprise-packages, and extra-packages,
3+
# and can be assumed to be the "source of truth" for our pre-commit rules.
4+
# See https://eng.inky.wtf/docs/technical/git/pre-commit for info on how to edit this file.
5+
repos:
6+
- repo: https://github.com/chainguard-dev/pre-commit-hooks
7+
rev: e4f3bba353cc583ce73f660dcf217e245fd681d3
8+
hooks:
9+
- id: check-for-epoch-bump
10+
- id: shellcheck-run-steps
11+
- repo: https://github.com/chainguard-dev/yam
12+
rev: 498642e77997ba79709f43a7ee2c84b12b2145bb # v0.2.12
13+
hooks:
14+
- id: yam
15+
# TODO: Swap with CG repo: - repo: https://github.com/wolfi-dev/wolfictl
16+
- repo: https://github.com/amberarcadia/wolfictl
17+
rev: 85b1301c4d17fcd0c8f0ce455724941cae815d68
18+
hooks:
19+
- id: wolfictl-lint
20+
- repo: https://github.com/golangci/misspell
21+
rev: 528d713e620bdf4b41849db93cb489c4fef9f5c5 # v0.6.0
22+
hooks:
23+
- id: misspell
24+
- repo: https://github.com/pre-commit/pre-commit-hooks
25+
rev: cef0300fd0fc4d2a87a85fa2093c6b283ea36f4b # v5.0.0
26+
hooks:
27+
- id: check-yaml
28+
- id: forbid-submodules
29+
- id: check-added-large-files
30+
- id: check-case-conflict
31+
- id: check-merge-conflict
32+
- id: check-symlinks
33+
- id: detect-private-key
34+
exclude: ^ruby-3\.0/0001-ruby-3\.0\.6-openssl-patch\.patch$

pre_commit_hooks/__init__.py

Whitespace-only changes.
Lines changed: 128 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,128 @@
1+
from __future__ import annotations
2+
3+
import argparse
4+
import contextlib
5+
import os
6+
import subprocess
7+
import tempfile
8+
from collections.abc import Generator
9+
from collections.abc import Mapping
10+
from collections.abc import Sequence
11+
from typing import Any
12+
from typing import NamedTuple
13+
14+
import ruamel.yaml
15+
16+
yaml = ruamel.yaml.YAML(typ="safe")
17+
18+
19+
def _exhaust(gen: Generator[str]) -> None:
20+
for _ in gen:
21+
pass
22+
23+
24+
def _parse_unsafe(*args: Any, **kwargs: Any) -> None:
25+
_exhaust(yaml.parse(*args, **kwargs))
26+
27+
28+
def _load_all(*args: Any, **kwargs: Any) -> None:
29+
_exhaust(yaml.load_all(*args, **kwargs))
30+
31+
32+
class Key(NamedTuple):
33+
multi: bool
34+
unsafe: bool
35+
36+
37+
def do_shellcheck(
38+
melange_cfg: Mapping[str, Any],
39+
shellcheck: list[str],
40+
) -> None:
41+
if melange_cfg == {}:
42+
return
43+
44+
pkgs = [melange_cfg]
45+
pkgs.extend(melange_cfg.get("subpackages", []))
46+
pipelines: list[Mapping[str, Any]] = []
47+
for pkg in pkgs:
48+
pipelines.extend(pkg.get("pipeline", []))
49+
if "test" in pkg.keys():
50+
test_pipeline = pkg["test"].get("pipeline", [])
51+
pipelines.extend(test_pipeline)
52+
name = melange_cfg["package"]["name"]
53+
all_run_files = []
54+
with contextlib.ExitStack() as stack:
55+
for step in pipelines:
56+
if "runs" not in step.keys():
57+
continue
58+
all_run_files.extend(
59+
[
60+
stack.enter_context(
61+
tempfile.NamedTemporaryFile(
62+
mode="w",
63+
prefix=name,
64+
dir=os.getcwd(),
65+
delete_on_close=False,
66+
),
67+
),
68+
],
69+
)
70+
for shfile in all_run_files:
71+
shfile.write(step["runs"])
72+
shfile.close()
73+
subprocess.check_call(
74+
["/usr/bin/shellcheck"]
75+
+ ["--shell=busybox", "--"]
76+
+ [os.path.basename(f.name) for f in all_run_files],
77+
cwd=os.getcwd(),
78+
)
79+
80+
81+
def main(argv: Sequence[str] | None = None) -> int:
82+
parser = argparse.ArgumentParser()
83+
parser.add_argument("filenames", nargs="*", help="Filenames to check.")
84+
parser.add_argument(
85+
"--shellcheck",
86+
default=[
87+
"docker",
88+
"run",
89+
f"--volume={os.getcwd()}:/mnt",
90+
"--rm",
91+
"-it",
92+
"koalaman/shellcheck:latest",
93+
],
94+
nargs="*",
95+
help="shellcheck command",
96+
)
97+
args = parser.parse_args(argv)
98+
99+
melange_cfg = {}
100+
for filename in args.filenames:
101+
with tempfile.NamedTemporaryFile(
102+
"w",
103+
delete_on_close=False,
104+
) as compiled_out:
105+
subprocess.check_call(
106+
[
107+
"melange",
108+
"compile",
109+
"--arch=x86_64",
110+
"--pipeline-dir=./pipelines",
111+
filename,
112+
],
113+
stdout=compiled_out,
114+
)
115+
compiled_out.close()
116+
try:
117+
with open(compiled_out.name) as compiled_in:
118+
melange_cfg = yaml.load(compiled_in)
119+
do_shellcheck(melange_cfg, args.shellcheck)
120+
except ruamel.yaml.YAMLError as exc:
121+
print(exc)
122+
return 1
123+
124+
return 0
125+
126+
127+
if __name__ == "__main__":
128+
raise SystemExit(main())

ruff.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
line-length = 80

setup.cfg

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
[metadata]
2+
name = pre_commit_hooks
3+
version = 0.0.1
4+
description = chainguard hooks for pre-commit
5+
long_description = file: README.md
6+
long_description_content_type = text/markdown
7+
url = https://github.com/chainguard-dev/pre-commit-hooks
8+
license = MIT
9+
license_files = LICENSE
10+
classifiers =
11+
Programming Language :: Python :: 3
12+
Programming Language :: Python :: 3 :: Only
13+
Programming Language :: Python :: Implementation :: CPython
14+
Programming Language :: Python :: Implementation :: PyPy
15+
16+
[options]
17+
packages = find:
18+
install_requires =
19+
ruamel.yaml>=0.15
20+
python_requires = >=3.9
21+
22+
[options.entry_points]
23+
console_scripts =
24+
shellcheck-run-steps = pre_commit_hooks.shellcheck_run_steps:main
25+
26+
[bdist_wheel]
27+
universal = True
28+
29+
[coverage:run]
30+
plugins = covdefaults
31+
32+
[mypy]
33+
check_untyped_defs = true
34+
disallow_any_generics = true
35+
disallow_incomplete_defs = true
36+
disallow_untyped_defs = true
37+
warn_redundant_casts = true
38+
warn_unused_ignores = true
39+
40+
[mypy-testing.*]
41+
disallow_untyped_defs = false
42+
43+
[mypy-tests.*]
44+
disallow_untyped_defs = false

setup.py

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
from __future__ import annotations
2+
3+
from setuptools import setup
4+
5+
setup()

0 commit comments

Comments
 (0)