597f76a753
The 6s per-file deadline + baseline=3981 were tuned to a fast dev machine; a slower CI runner could time out a sub-second finite file, dropping total-pass below baseline and flaking CI red. The deadline is now an env var (default 6 locally); CI sets it to 20s for headroom. Characterized the 6 timeouts: identical pass/timeout counts at 6s and 20s, so all 6 are genuinely-infinite hangs (killed at any deadline) and the 227 finite files finish well under 6s. So no baseline margin is needed — a generous CI deadline removes the flake risk while preserving full regression sensitivity. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
57 lines
1.8 KiB
YAML
57 lines
1.8 KiB
YAML
name: tests
|
|
|
|
# Run the full test suite (jpm test) on every push and pull request.
|
|
on:
|
|
push:
|
|
pull_request:
|
|
|
|
jobs:
|
|
test:
|
|
runs-on: ubuntu-latest
|
|
env:
|
|
JANET_VERSION: v1.41.2 # bump to match the version Jolt is developed against
|
|
# Per-file deadline for the clojure-test-suite battery. Finite files finish
|
|
# in well under 1s; the genuinely-infinite ones get killed at any deadline.
|
|
# A generous value gives slow CI runners headroom so a sub-second file
|
|
# spiking doesn't time out and drop total-pass below the baseline.
|
|
JOLT_SUITE_TIMEOUT: "20"
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
with:
|
|
# Submodules: vendor/sci (SCI bootstrap/runtime tests) and
|
|
# vendor/clojure-test-suite (the cross-dialect conformance battery,
|
|
# asserted against a baseline by clojure-test-suite-test.janet).
|
|
submodules: recursive
|
|
|
|
- name: Cache Janet build
|
|
id: cache-janet
|
|
uses: actions/cache@v4
|
|
with:
|
|
path: /tmp/janet
|
|
key: janet-${{ env.JANET_VERSION }}-${{ runner.os }}
|
|
|
|
- name: Build Janet
|
|
if: steps.cache-janet.outputs.cache-hit != 'true'
|
|
run: |
|
|
git clone --depth 1 --branch "$JANET_VERSION" https://github.com/janet-lang/janet.git /tmp/janet
|
|
make -C /tmp/janet
|
|
|
|
- name: Install Janet
|
|
run: sudo make -C /tmp/janet install
|
|
|
|
- name: Install jpm
|
|
run: |
|
|
git clone --depth 1 https://github.com/janet-lang/jpm.git /tmp/jpm
|
|
# bootstrap.janet resolves jpm/cli.janet relative to the cwd, so it
|
|
# must run from inside the jpm checkout.
|
|
cd /tmp/jpm
|
|
sudo janet bootstrap.janet
|
|
|
|
- name: Janet version
|
|
run: janet -v
|
|
|
|
- name: Build executable
|
|
run: jpm build
|
|
|
|
- name: Run tests
|
|
run: jpm test
|