Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Android rebase #202

Open
wants to merge 531 commits into
base: android
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
531 commits
Select commit Hold shift + click to select a range
82bf44a
Add ability to remove expired status reports
tysmith Jun 24, 2021
fbb8297
Remove imports in common/__init__.py to avoid circular import issues
tysmith Jul 7, 2021
3ec3d36
Add Target assets
tysmith Jul 8, 2021
4da33fd
Increase support for assets
tysmith Jul 14, 2021
a9be0b0
[tests] Use `spec_set` with Mock where possible
tysmith Jul 15, 2021
3ba5170
Allow updating existing assets
tysmith Jul 16, 2021
a64fc17
Move AssetManager to assets.py
tysmith Jul 19, 2021
bfceeea
Replace Target.add_abort_token() with asset
tysmith Jul 19, 2021
1dabb0a
[ci] Update pinned tool versions
tysmith Jul 19, 2021
c3adf7d
Move environment handling to Target
tysmith Jul 23, 2021
06d8488
Fix '--prefs' argument
tysmith Jul 27, 2021
e628d0c
TestCase is not responsible for calling AssetManager.cleanup()
tysmith Jul 27, 2021
d0f0d97
Include assets in reduced test cases
tysmith Jul 28, 2021
2296b15
[test] Add coverage and comment to storage.py
tysmith Jul 29, 2021
493d5b3
[sapphire] Use enum for served status codes
tysmith Aug 6, 2021
556ad6c
Use enum for Target Result codes
tysmith Aug 6, 2021
e1dadc1
Rename Target.detect_failure() -> Target.check_result()
tysmith Aug 6, 2021
113180d
Use enum for Exit codes
tysmith Aug 7, 2021
f19ec13
Use enum for testcase quality
tysmith Aug 7, 2021
08fbfb2
Use enum for stack hasher modes
tysmith Aug 9, 2021
7b4032f
Minor refactor of TargetMonitor.log_data()
tysmith Aug 9, 2021
4e6829d
[ci] Bump pinned pylint version
tysmith Aug 9, 2021
3e9830a
FM CrashEntry.testcase_quality is an int
tysmith Aug 11, 2021
ca75f61
[reduce] Display initial quality in grizzly.reduce.crash
tysmith Aug 11, 2021
41ddbf8
[reduce] Update comments to use quality name instead of Q#
tysmith Aug 11, 2021
0797876
Add Target.filtered_environ()
tysmith Aug 17, 2021
44910d0
Use IntEnum for Exit and Quality
tysmith Aug 17, 2021
bfcb7cc
[tests] Add coverage of Target.filtered_environ()
tysmith Aug 18, 2021
1c68658
Pass integer Quality value to FM
tysmith Aug 18, 2021
74e681c
Simplify filtered_environ()
tysmith Sep 9, 2021
5ebb2a6
Check signature cache before creating a signature
tysmith Sep 10, 2021
15f6439
Use signature hash instead of description to bucket and count results
tysmith Sep 13, 2021
cf9f3a6
Move Report to a new file
tysmith Sep 14, 2021
c4dab64
Use sqlite database to store status data
tysmith Sep 20, 2021
4eaa389
Use ResultCounter in Status to count results that have been found
tysmith Sep 23, 2021
82a19c4
Move db version checking to a function
tysmith Sep 28, 2021
dbca0b4
Require minimum two local results before is_frequent() can return True
tysmith Sep 29, 2021
9635752
Add '--limit-reports' command line argument
tysmith Sep 30, 2021
d3a727a
Remove report limiting from FuzzManagerReporter
tysmith Oct 1, 2021
c55502e
Set default report limit to 5
tysmith Oct 1, 2021
03a9a20
Automate layout of summary status reports
tysmith Oct 1, 2021
ba80e63
Add ability to detect potential blocking results
tysmith Oct 2, 2021
10f72b3
Warn in summary report if potential blockers are detected
tysmith Oct 2, 2021
00ace68
Fix status db update race
tysmith Oct 5, 2021
99be7e2
Tighten up db version check assertions
tysmith Oct 5, 2021
c6ac565
Update status reporter command line flags
tysmith Oct 5, 2021
aeb86ea
Only lock db when update is required
tysmith Oct 7, 2021
10f21ca
Include both logical and physical cores in summary status report
tysmith Oct 7, 2021
6cd4858
Handle FM signature creation failures
tysmith Oct 8, 2021
0f52440
[tests] Update patch missed when splitting file
tysmith Oct 8, 2021
a2673d6
Make ResultCounter.load() batch load from db
tysmith Oct 12, 2021
ea0216f
Avoid status data pid reuse
tysmith Oct 12, 2021
78943bc
Properly close status db connection
tysmith Oct 12, 2021
81b32f7
Reuse existing db connection in _db_version_check()
tysmith Oct 12, 2021
e047fa0
Add 'PRIMARY KEY's to tables in status db
tysmith Oct 12, 2021
063be3f
A result must be seen more than once before it can be considered a bl…
tysmith Oct 12, 2021
a902d84
[tests] Add missing StatusReporter._summary() coverage
tysmith Oct 13, 2021
b6c027a
Reintroduce FuzzManagerReporter.force_crash
tysmith Oct 15, 2021
949d35d
Set DB_TIMEOUT=30 to avoid 'database is locked' exception on busy system
tysmith Oct 15, 2021
bc0ce0e
Use contextlib.closing() with sqlite3.connect()
tysmith Oct 15, 2021
ae48fc0
Remove unnecessary db checks when loading status data
tysmith Oct 18, 2021
07ed527
Avoid db lookup if local result is frequent
tysmith Oct 18, 2021
34990a4
Call status.report() when session is complete
tysmith Oct 18, 2021
b0b3551
Update full status report output
tysmith Oct 19, 2021
ba20928
Add ResultEntry
tysmith Oct 19, 2021
c433295
Remove unnecessary wrapper functions from status reporter
tysmith Oct 19, 2021
e370b2b
[reducer] Add original crash ID to FuzzManager metadata.
jschwartzentruber Oct 19, 2021
1206b2d
[reducer] Do not include stats in unexpected results
tysmith Oct 20, 2021
2242fc1
Add StatusReporter.format_entries()
tysmith Oct 20, 2021
01d6629
[sapphire] Pass query to dynamic response callback
tysmith Nov 2, 2021
70e0901
Add ability to display log when submitting report
tysmith Nov 2, 2021
2595240
Simplify TestCase and TestFile implementations
tysmith Nov 8, 2021
c46b824
[sapphire] Update Job.served to yield posix paths
tysmith Nov 9, 2021
38515e7
Update TestCase to use Path objects where possible
tysmith Nov 9, 2021
65c4cd8
Move files unpacked from a testcase archive
tysmith Nov 9, 2021
bec3484
[replay] Remove use of TestCase.dump() to run tests
tysmith Nov 9, 2021
6452000
Serve TestCases directly from data path
tysmith Nov 9, 2021
cd7c7c8
Simplify TestFile by making it a namedtuple
tysmith Nov 11, 2021
c6b63d6
[tests] Use Path() where possible in test_storage.py
tysmith Nov 13, 2021
1673dd5
[tests] Cleanup properly after tests
tysmith Nov 13, 2021
a438efe
Use shutil.copyfile() and shutil.move() and remove TestCase.copy_file()
tysmith Nov 13, 2021
9f36d42
Avoid marking files that might not be served as required
tysmith Nov 15, 2021
ecb3e76
Move STATUS_DB to Status.
jschwartzentruber Nov 5, 2021
bf9de93
Add reducer status reporting.
jschwartzentruber Nov 5, 2021
743b806
Add an iteration callback to replay, and use it in reduce.
jschwartzentruber Nov 17, 2021
020377c
Use global temp in Tox to speed up tests.
jschwartzentruber Nov 17, 2021
90662d7
[sapphire] Simplify and rename check_request() -> lookup_resource()
tysmith Nov 19, 2021
1dbf472
[tests] Update Sapphire test client path handling
tysmith Nov 21, 2021
eb9bf5a
[sapphire] Fix redirecting to files with non-ASCII file names
tysmith Nov 21, 2021
d5b36dc
[sapphire] Fix requesting includes with non-ASCII file names
tysmith Nov 22, 2021
8d4e229
[sapphire] Re-order some operations in handle_request()
tysmith Nov 22, 2021
9fe18b9
[sapphire] Update access check
tysmith Nov 22, 2021
91b052d
Make analysis line more concise in reducer status.
jschwartzentruber Nov 18, 2021
6c8deb1
Always write reducer stats to DB when recording a milestone.
jschwartzentruber Nov 18, 2021
907153e
Add Crash ID to final reduction report.
jschwartzentruber Nov 18, 2021
390eb1e
Show tool name in reducer status report.
jschwartzentruber Nov 18, 2021
62cda53
Include latest reported crashes in reduction status.
jschwartzentruber Nov 18, 2021
2f332a0
Filter file list passed to TestCase.add_batch()
tysmith Nov 23, 2021
89f7cb5
Fix support for absolute and relative paths in TestCase.purge_optional()
tysmith Nov 23, 2021
b33ff35
[sapphire] Use Path() where possible in Job()
tysmith Nov 25, 2021
3d09e5a
[sapphire] Have Job() only accept a Path()
tysmith Nov 25, 2021
a90ae59
[sapphire] Remove Job.initial_queue_size
tysmith Nov 25, 2021
7c08f63
Add '--smoke-test' command line argument
tysmith Nov 26, 2021
1527917
[tests] Patch common.report.Collector to avoid scanning disk for sign…
tysmith Nov 26, 2021
efefb96
Make '--smoke-test' display browser crash log
tysmith Nov 26, 2021
83d87d5
Add testcase Quality.IGNORED
tysmith Nov 26, 2021
79a260a
Update minimum required version of FFPuppet
tysmith Nov 26, 2021
b36b7eb
Fix typo
tysmith Nov 30, 2021
fca0e4d
Add 'grizzly.replay.crash' and 'grizzly.replay.bucket'
tysmith Nov 30, 2021
1c41478
[tests] Add on_iteration_cb coverage
tysmith Dec 2, 2021
33f27ad
Report other results after each reduction stage, rather than
jschwartzentruber Jan 7, 2022
2dcea8f
Update '--logs' argument
tysmith Jan 21, 2022
1985837
Add GRZ_TMP environment variable to override use of os.gettempdir() i…
tysmith Jan 21, 2022
596318e
[ci] Add macOS CI
jschwartzentruber Jan 20, 2022
0eecde2
Fix a few typos
tysmith Jan 27, 2022
9422546
Round the results of getloadavg() since precision varies across platf…
tysmith Jan 28, 2022
8ad4a4f
[sapphire] Use Path.rglob() instead of os.walk()
tysmith Feb 1, 2022
bf4b880
Replace another instance of os.walk() with Path.rglob()
tysmith Feb 1, 2022
5788305
Use pathlib in a few more places
tysmith Feb 1, 2022
5a31074
Switch to pathlib in adapter.py
tysmith Feb 1, 2022
5fa647a
Fix spacing in 'CPU & Load' reporting
tysmith Feb 1, 2022
22074c5
Use Path objs in a few more places
tysmith Feb 1, 2022
547d84c
Check '/proc/sys/kernel/perf_event_paranoid' before launching with rr
tysmith Feb 2, 2022
35ff2a6
Use pathlib in replay
tysmith Feb 4, 2022
b1d912c
[sapphire] Include query string in redirect
tysmith Feb 11, 2022
82769f8
Use common entry point '/grz_start'
tysmith Feb 11, 2022
a3a20d4
[replay] Display message before processing result
tysmith Feb 22, 2022
19d96cf
Use new PrefPicker API
tysmith Mar 1, 2022
d2bb6ed
Update argument handling
tysmith Feb 27, 2022
5ead9e9
[ci] Update flake8 ignore list
tysmith Mar 3, 2022
ab92b89
[ci] Add Python3.10
tysmith Mar 3, 2022
820d0d3
[ci] Update tools used by pre-commit
tysmith Mar 3, 2022
cc213f2
[tests] Remove unused logger
tysmith Mar 3, 2022
ee1c682
[ci] Add .codecov.yml
tysmith Mar 3, 2022
902bf9f
[tests] Fix test_report_counter_02() on Python3.10
tysmith Mar 3, 2022
8de7c5a
Handle startup crashes properly when using '--any-crash'
tysmith Mar 11, 2022
5669bd7
Avoid closing the browser before finishing test
tysmith Mar 18, 2022
3434c14
Remove test time limit timer after test tab closes
tysmith Mar 18, 2022
bc4b605
[replay] Require --logs with --pernosco or --rr
tysmith Mar 25, 2022
a73048e
[ci] Pin click version used by black
tysmith Mar 30, 2022
7229c03
[harness] Remove multiple return statements from main()
tysmith Apr 1, 2022
38e6dc3
[sapphire] Track total number of workers launched
tysmith Apr 11, 2022
356d141
[sapphire] Call socket.shutdown() when closing active workers
tysmith Apr 11, 2022
d85574e
Update README
tysmith Apr 25, 2022
3da0c12
[reduce] Add argument --no-repro-quality
tysmith Apr 27, 2022
7107077
[reduce] Quality args should be ints
tysmith Apr 27, 2022
88bc878
Remove lower limit from launch timeout
tysmith Apr 29, 2022
12b0c2d
[reduce] Normalize line-endings to \n before beautification.
jschwartzentruber May 11, 2022
692f00e
[ci] Update macOS workers to use macos-latest
tysmith Jul 27, 2022
f41f4ac
Remove an invalid assertion
tysmith Jul 20, 2022
08924f1
[ci] Update Black to 22.6.0
tysmith Jul 28, 2022
1dac7c1
Add support for built in browser headless mode
tysmith Jul 28, 2022
f5f758f
Avoid sending SIGUSR1 to debugger when running with --coverage
tysmith Aug 9, 2022
2992569
Use wait_procs() after sending SIGABRT to hung process
tysmith Aug 9, 2022
b39a72e
Only send SIGUSR1 to browser processes when using --coverage
tysmith Aug 10, 2022
fd70361
Only scan for open gcda files if SIGUSR1 was sent
tysmith Aug 10, 2022
c1b89e9
Increase timeouts in dump_coverage()
tysmith Aug 10, 2022
7a18dde
Fix process executable name check in dump_coverage()
tysmith Aug 19, 2022
1dd39eb
Add --launch-attempts argument
tysmith Sep 2, 2022
9e13457
[replay] Add --post-launch-delay argument
tysmith Sep 8, 2022
fca2ce4
[sapphire] Wait for window.onload on 404 page
tysmith Sep 12, 2022
494bbc2
[harness] Use requestIdleCallback to close harness if available
tysmith Sep 12, 2022
e2786bf
Fix missing results in status reports
tysmith Sep 13, 2022
14c4d6d
[tests] Purge expired Status entries
tysmith Sep 13, 2022
59f3c7e
Add Android support
tysmith May 1, 2019
81dfcfa
Add adb target/device launch timeout
tysmith May 29, 2019
93b0517
Fixes for --install-asan
jschwartzentruber Jun 7, 2019
922f8ba
Update adb device tests and split up files
tysmith Jul 24, 2019
c0ed671
[tests] Fix adb tests
tysmith Jul 24, 2019
5080e80
Remove duplicate code
tysmith Jul 25, 2019
b640e6d
[tests] Fix Python3 test failures
tysmith Jul 25, 2019
1e3a41f
[tests] Update adb_target tests
tysmith Jul 25, 2019
342822c
Refactor checks for required binaries
tysmith Jul 25, 2019
bc3758a
[tests] Add get_enforce() and set_enforce() tests
tysmith Jul 25, 2019
c924258
Remove calls to shell stop/start from set_enforce()
tysmith Jul 26, 2019
2debf78
Add 'prep' option to device.py
tysmith Jul 26, 2019
276dd7e
[tests] Increase adb_session.py test coverage
tysmith Jul 26, 2019
88dd54c
[tests] Add ADBProcess.save_logs() tests
tysmith Jul 27, 2019
7392973
[tests] Add ADBProcess._process_logs() tests
tysmith Jul 27, 2019
80b87c0
Revert aapt path
tysmith Jul 27, 2019
b14ab1c
[adb_device] Uninstall existing APK before installing APK
tysmith Jul 31, 2019
5014050
Cleanup ADBSession.connect()
tysmith Jul 31, 2019
37459ca
[tests] Increase test coverage of adb_session.py
tysmith Jul 31, 2019
d0fbc71
Update ADBProcess._split_logcat() and add test coverage
tysmith Aug 1, 2019
4e6f8aa
Remove extra profile upload code
tysmith Aug 1, 2019
5c236e7
Rename DeviceProcess to DeviceProcessInfo
tysmith Aug 1, 2019
1d4cf16
Remove fake_adb.py
tysmith Aug 1, 2019
ac0b160
[tests] Fix test_adb_process_10 on Windows
tysmith Aug 1, 2019
148db6c
Make logcat parsing more robust
tysmith Aug 2, 2019
44e40f8
Make ADBSession.connect() more robust
tysmith Aug 2, 2019
741e347
Make device detection more robust
tysmith Aug 14, 2019
7bd3a3c
Add retry to bottstrap port reverse attempt
tysmith Aug 15, 2019
b1ab1d5
Misc nits
tysmith Aug 16, 2019
a35d22d
Add reboot_device() and remount()
tysmith Aug 16, 2019
6e08ec0
Use any() instead of bool() on output of a generator
tysmith Aug 16, 2019
14e28b5
Add ASan support
tysmith Aug 21, 2019
ddf90b4
Make logcat scanning more robust
tysmith Aug 21, 2019
8b7ceed
Better filtering of ASan output
tysmith Aug 21, 2019
bd093c6
Add more ASAN_OPTIONS
tysmith Aug 21, 2019
91ee9f0
Fix a couple nits
tysmith Aug 21, 2019
c2f25cc
Reorganize ADBSession
tysmith Sep 3, 2019
4ae6ad2
Make DeviceProcessInfo a namedtuple
tysmith Sep 3, 2019
4fc0a40
Fix log mode for Python 3
jschwartzentruber Sep 20, 2019
c4e257b
No need to set permissions: `adb install -g` grants them all.
jschwartzentruber Sep 20, 2019
f236209
Fix tests broken in 536a7878f5cd5302ffbc9502f4c0f049ef4b5b2d
jschwartzentruber Sep 20, 2019
c09ed7c
[android] Add PrefPicker support
tysmith Aug 10, 2020
498dbca
Fix copy/paste error
tysmith Aug 10, 2020
780e8ef
[android] Fix geckoview_example launch
tysmith Aug 10, 2020
adb2e6c
[adb] Check package is not running before launching
tysmith Aug 12, 2020
7e311f9
[adb] Remove install ASan functionality
tysmith Aug 12, 2020
0b81a4e
[adb] Add ADBSession.sanitizer_options()
tysmith Aug 12, 2020
caca204
[adb] Fix call to install_file()
tysmith Aug 12, 2020
bb6e87f
[adb] Update handling sanitizer logs
tysmith Aug 12, 2020
8c9654b
[adb] Add support for geckoview-config.yaml
tysmith Aug 14, 2020
258142f
[tests] Fix and speedup ADBSession tests
tysmith Aug 14, 2020
be20960
[adb] Add __slots__ to ADBProcess and ADBSession
tysmith Aug 14, 2020
c8a6fbb
[android] Fix Target and Bootstrapper after rebase
jschwartzentruber Feb 4, 2021
72e4207
[android] Update AAPT/ADB search to use updated SDK locations.
jschwartzentruber Feb 4, 2021
6b4965f
[android] Remove log_path from ASAN_OPTIONS
jschwartzentruber Feb 8, 2021
def2c89
[adb] Update bit-rotted code
tysmith Feb 17, 2022
2cdb689
[adb] Apply linter fixes and nits
tysmith Feb 18, 2022
e4f6252
[adb] Use new PrefPicker API
tysmith Mar 8, 2022
f8d92dc
Make pref handling more robust
tysmith Mar 12, 2022
29c0f8a
[adb] Place profile in '/data/local/tmp/'
tysmith Mar 14, 2022
4b75be9
[adb] Make prefs_to_dict() return None instead of raising
tysmith Mar 14, 2022
6f3cb3a
[adb] Use Enum for Reason code
tysmith Mar 14, 2022
f676525
[adb] Split main() and parse_args() into separate functions
tysmith Mar 15, 2022
f946355
[tests] Fix adb tests on Windows
tysmith Mar 16, 2022
df046a1
[tests] Increase coverage of adb components
tysmith Mar 17, 2022
b896535
Use relative imports
tysmith Mar 17, 2022
a4b04db
[adb] Add missing header comments
tysmith Mar 17, 2022
ec149c4
[adb] Rename device.py -> main.py
tysmith Mar 17, 2022
139eed9
[adb] Update support for ASan builds
tysmith Mar 23, 2022
55624c7
[adb] Scan wrap.sh output for ASan logs
tysmith Mar 23, 2022
0d0390e
[tests] Update ADBSession tests
tysmith Mar 23, 2022
c6e20d3
[adb] Raise TargetLaunchError and TargetLaunchTimeout from ADBTarget.…
tysmith Mar 23, 2022
3957ba6
[adb] Handle ADBLaunchErrors in target
tysmith Mar 28, 2022
42ea47e
[adb] Add ADBSession.shell()
tysmith Mar 30, 2022
ff2b07f
[adb] Use subprocess.run() in ADBSession._call_adb()
tysmith Mar 30, 2022
0698e53
[adb] Disable some UI animations in --prep
tysmith Mar 30, 2022
c4b9996
[tests] Fix test_adb_session_29
tysmith Mar 31, 2022
9386ab2
[adb] Don't try to return output when ADB call times out
tysmith Mar 31, 2022
4a20e0a
[adb] Add timeout to ADB calls
tysmith Apr 1, 2022
d276901
[adb] Add retry_delay kwarg to ADBSession.create()
tysmith Apr 1, 2022
4c26008
[adb] Refactor ADBSession._get_procs()
tysmith Apr 1, 2022
ac0cf13
[adb] Cleanup ADBSession.open_files()
tysmith Apr 4, 2022
6f00ccb
[adb] Only pass -A to ps when pid or --ppid is not provided
tysmith Apr 8, 2022
2a15ca8
[adb] Use pidof to lookup pid
tysmith Apr 8, 2022
3c29ed9
[adb] Simplify ps called used in process_exists()
tysmith Apr 8, 2022
c14be56
[adb] Update use of process_minidumps()
tysmith Apr 14, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .codecov.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
codecov:
ci:
- community-tc.services.mozilla.com
58 changes: 58 additions & 0 deletions .github/scripts/start-worker.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
#!/bin/sh -e

[ -z "$TASKCLUSTER_ACCESS_TOKEN" ] && echo "Missing TASKCLUSTER_ACCESS_TOKEN" >&2 && exit 2
[ -z "$TC_WORKER_ID" ] && echo "Missing TC_WORKER_ID" >&2 && exit 2

set -x

TC_VERSION=v44.4.0
TC_PROJECT=fuzzing
TC_WORKER_TYPE=ci-osx
TC_IDLE_TIMEOUT=300

TASKCLUSTER_ROOT_URL="https://community-tc.services.mozilla.com"
TASKCLUSTER_CLIENT_ID="project/$TC_PROJECT/worker-$TC_WORKER_TYPE-gh"

set +x
cat > worker.config <<EOF
{
"accessToken": "$TASKCLUSTER_ACCESS_TOKEN",
"clientId": "$TASKCLUSTER_CLIENT_ID",
"disableReboots": true,
"ed25519SigningKeyLocation": "worker.key",
"idleTimeoutSecs": $TC_IDLE_TIMEOUT,
"livelogExecutable": "$PWD/livelog",
"provisionerId": "proj-$TC_PROJECT",
"publicIP": "127.0.0.1",
"requiredDiskSpaceMegabytes": 512,
"rootURL": "$TASKCLUSTER_ROOT_URL",
"sentryProject": "generic-worker",
"taskclusterProxyExecutable": "$PWD/taskcluster-proxy",
"taskclusterProxyPort": 8080,
"tasksDir": "tasks",
"workerGroup": "proj-$TC_PROJECT",
"workerId": "$TC_WORKER_ID",
"workerType": "$TC_WORKER_TYPE",
"wstAudience": "communitytc",
"wstServerURL": "https://community-websocktunnel.services.mozilla.com"
}
EOF
set -x
unset TASKCLUSTER_ACCESS_TOKEN

curl -sSL "https://github.com/taskcluster/taskcluster/releases/download/$TC_VERSION/generic-worker-simple-darwin-amd64" -o generic-worker
curl -sSL "https://github.com/taskcluster/taskcluster/releases/download/$TC_VERSION/livelog-darwin-amd64" -o livelog
curl -sSL "https://github.com/taskcluster/taskcluster/releases/download/$TC_VERSION/taskcluster-proxy-darwin-amd64" -o taskcluster-proxy
chmod +x generic-worker livelog taskcluster-proxy

./generic-worker new-ed25519-keypair --file worker.key
mkdir tasks
set +e
./generic-worker run --config worker.config
case $? in
0|68)
;;
*)
exit $?
;;
esac
11 changes: 11 additions & 0 deletions .github/workflows/taskcluster-pr.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
name: taskcluster PR worker
on: [pull_request]
jobs:
osx-tc-worker-pr:
runs-on: macos-latest
env:
TASKCLUSTER_ACCESS_TOKEN: ${{ secrets.TASKCLUSTER_ACCESS_TOKEN }}
TC_WORKER_ID: "gh-grizzly-pr-${{ github.run_id }}-${{ github.run_attempt }}"
steps:
- uses: actions/checkout@v2
- run: .github/scripts/start-worker.sh
11 changes: 11 additions & 0 deletions .github/workflows/taskcluster-push.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
name: taskcluster push worker
on: [push]
jobs:
osx-tc-worker-push:
runs-on: macos-latest
env:
TASKCLUSTER_ACCESS_TOKEN: ${{ secrets.TASKCLUSTER_ACCESS_TOKEN }}
TC_WORKER_ID: "gh-grizzly-push-${{ github.run_id }}-${{ github.run_attempt }}"
steps:
- uses: actions/checkout@v2
- run: .github/scripts/start-worker.sh
11 changes: 11 additions & 0 deletions .github/workflows/taskcluster-release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
name: taskcluster release worker
on: [release]
jobs:
osx-tc-worker-release:
runs-on: macos-latest
env:
TASKCLUSTER_ACCESS_TOKEN: ${{ secrets.TASKCLUSTER_ACCESS_TOKEN }}
TC_WORKER_ID: "gh-grizzly-release-${{ github.run_id }}-${{ github.run_attempt }}"
steps:
- uses: actions/checkout@v2
- run: .github/scripts/start-worker.sh
60 changes: 60 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
repos:
- repo: https://github.com/pycqa/isort
rev: 5.10.1
hooks:
- id: isort
- repo: https://github.com/asottile/yesqa
rev: v1.3.0
hooks:
- id: yesqa
- repo: https://github.com/ambv/black
rev: 22.6.0
hooks:
- id: black
- repo: https://gitlab.com/pycqa/flake8
rev: 4.0.1
hooks:
- id: flake8
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.1.0
hooks:
- id: check-ast
- id: check-docstring-first
- id: check-executables-have-shebangs
- id: check-merge-conflict
- id: check-symlinks
- id: debug-statements
- id: trailing-whitespace
- id: check-yaml
- id: mixed-line-ending
- id: name-tests-test
args: ['--django']
- id: check-json
- id: requirements-txt-fixer
- repo: https://github.com/codespell-project/codespell
rev: v2.1.0
hooks:
- id: codespell
exclude_types: [json]
- repo: https://github.com/marco-c/taskcluster_yml_validator
rev: v0.0.7
hooks:
- id: taskcluster_yml
- repo: https://github.com/MozillaSecurity/orion
rev: v0.0.4
hooks:
- id: orion_ci
- repo: meta
hooks:
- id: check-useless-excludes
- repo: local
hooks:
- id: pylint
name: pylint
entry: tox -e pylint --
language: system
require_serial: true
types: [python]

default_language_version:
python: python3
16 changes: 0 additions & 16 deletions .pylintrc

This file was deleted.

134 changes: 134 additions & 0 deletions .taskcluster.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
version: 1
policy:
pullRequests: collaborators
tasks:
$let:
user: ${event.sender.login}

fetch_rev:
$if: 'tasks_for == "github-pull-request"'
then: ${event.pull_request.head.sha}
else:
$if: 'tasks_for == "github-push"'
then: ${event.after}
else: ${event.release.tag_name}

fetch_ref:
$if: 'tasks_for == "github-pull-request"'
then: ${event.pull_request.head.sha}
else:
$if: 'tasks_for == "github-push"'
then: ${event.after}
else: "refs/tags/${event.release.tag_name}:refs/tags/${event.release.tag_name}"

http_repo:
$if: 'tasks_for == "github-pull-request"'
then: ${event.pull_request.base.repo.clone_url}
else: ${event.repository.clone_url}

codecov_secret:
codecov-grizzly

pypi_secret:
pypi-grizzly

project_name:
Grizzly

matrix:
language: python
secrets:
- type: env
secret: project/fuzzing/codecov-grizzly
name: CODECOV_TOKEN
key: token
script:
- bash
- '-xec'
- tox; tox -e codecov
jobs:
include:
- name: tests python 3.6
version: "3.6"
env:
TOXENV: py36,lint
- name: tests python 3.7
version: "3.7"
env:
TOXENV: py37,lint
- name: tests python 3.8
version: "3.8"
env:
TOXENV: py38,lint
- name: test python 3.8 (windows)
version: "3.8"
platform: windows
env:
TOXENV: py38
- name: tests python 3.9
version: "3.9"
env:
TOXENV: py39,lint
- name: tests python 3.9 (macos)
version: "3.9"
platform: macos
env:
TOXENV: py39
- name: tests python 3.10
version: "3.10"
env:
TOXENV: py310,lint
- name: PyPI upload
version: "3.8"
env:
TOXENV: pypi
script:
- tox
when:
release: true
all_passed: true
secrets:
- type: env
secret: project/fuzzing/pypi-grizzly
name: TWINE_USERNAME
key: username
- type: env
secret: project/fuzzing/pypi-grizzly
name: TWINE_PASSWORD
key: password

in:
$if: 'tasks_for in ["github-push", "github-release"] || (tasks_for == "github-pull-request" && event["action"] in ["opened", "reopened", "synchronize"])'
then:
- created: {$fromNow: ''}
deadline: {$fromNow: '1 hour'}
provisionerId: proj-fuzzing
workerType: ci
payload:
features:
taskclusterProxy: true
maxRunTime: 3600
image:
type: indexed-image
path: public/orion-decision.tar.zst
namespace: project.fuzzing.orion.orion-decision.master
env:
PROJECT_NAME: ${project_name}
CI_MATRIX: {$json: {$eval: matrix}}
GITHUB_EVENT: {$json: {$eval: event}}
GITHUB_ACTION: ${tasks_for}
TASKCLUSTER_NOW: ${now}
command:
- ci-decision
- -v
scopes:
- queue:create-task:highest:proj-fuzzing/ci
- queue:create-task:highest:proj-fuzzing/ci-*
- queue:scheduler-id:taskcluster-github
- secrets:get:project/fuzzing/codecov-grizzly
- secrets:get:project/fuzzing/pypi-grizzly
metadata:
name: ${project_name} CI decision
description: Schedule CI tasks for ${project_name}
owner: [email protected]
source: https://github.com/MozillaSecurity/grizzly
40 changes: 0 additions & 40 deletions .travis.yml

This file was deleted.

4 changes: 2 additions & 2 deletions CODE_OF_CONDUCT.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Community Participation Guidelines

This repository is governed by Mozilla's code of conduct and etiquette guidelines.
This repository is governed by Mozilla's code of conduct and etiquette guidelines.
For more details, please read the
[Mozilla Community Participation Guidelines](https://www.mozilla.org/about/governance/policies/participation/).
[Mozilla Community Participation Guidelines](https://www.mozilla.org/about/governance/policies/participation/).

## How to Report
For more information on how to report violations of the Community Participation Guidelines, please read our '[How to Report](https://www.mozilla.org/about/governance/policies/participation/reporting/)' page.
Expand Down
1 change: 1 addition & 0 deletions MANIFEST.in
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
include grizzly/common/harness.html
Loading