Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

QA Updates - Week 9-10 of Q4 #43

Merged
merged 10 commits into from
Dec 3, 2024
4 changes: 2 additions & 2 deletions content/qa/ift/projects-qa-strategy-2024q4.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ identify gaps, and provide recommendations for improvements.

* fully qualified name: `vac:qa:ift:projects-qa-strategy-2024q4:status`
* owner: fbarbu15
* status: in progress(50%)
* status: done
* start-date: 2024/11/04
* end-date: 2024/11/22

Expand All @@ -74,7 +74,7 @@ Investigate and document Status QA strategy,
identify gaps, and provide recommendations for improvements.

#### Deliverables
* Link with Notion page
* Status Test Coverage [report](https://www.notion.so/Status-VAC-QA-Testing-Coverage-Report-14b8f96fb65c8086afc9df108bc170cc)

### Keycard

Expand Down
23 changes: 16 additions & 7 deletions content/qa/nomos/da-testing-2024q4.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,16 +73,24 @@ Issues Found:

* fully qualified name: `vac:qa:nomos:da-testing-2024q4:dispersal-storage-tests`
* owner: romanzac
* status: in progress (15%)
* status: done
* start-date: 2024/10/21
* end-date: 2024/11/04

#### Description
Add integration tests focusing on the dispersal and storage interface.

#### Deliverables
* PR link with added tests
* CI runs showing successful or failed executions with detailed reports
PRs:
- [PR #921](https://github.com/logos-co/nomos-node/pull/921)
- TC2.2: Ensure that DA nodes hold chunks in memory until the block with related BlobInfo is received.
- TC2.3: Confirm that attestations about received chunks are correctly sent back to the Zone.

CI Runs:
- [Run #12066967078](https://github.com/logos-co/nomos-node/actions/runs/12066967078)

Issues Found:
- [Unwrap on non-existent blob_col_idx](https://github.com/logos-co/nomos-node/issues/922)

### Certification and Verification Tests

Expand All @@ -103,7 +111,7 @@ Add integration tests focusing on certificate completeness.

* fully qualified name: `vac:qa:nomos:da-testing-2024q4:data-integrity-tests`
* owner: romanzac
* status: not started
* status: in progress (5%)
* start-date: 2024/11/22
* end-date: 2024/12/02

Expand All @@ -119,11 +127,12 @@ to reconstruct or retrieve data.

* fully qualified name: `vac:qa:nomos:da-testing-2024q4:edge-cases`
* owner: romanzac
* status: not started
* start-date: 2024/12/02
* end-date: 2024/12/20
* status: in progress (50%)
* start-date: 2024/10/01
* end-date: 2024/12/31

#### Description
This is an ongoing task.
Add edge case integration tests to cover unusual system behavior.
Test edge cases for behaviours implemented for dispersal, replication and sampling.

Expand Down
60 changes: 56 additions & 4 deletions content/qa/waku/interop-testing-2024q4.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,8 +137,8 @@ Issues found:
### Faster Runners

* fully qualified name: `vac:qa:waku:interop-testing-2024q4:faster-runners`
* owner: fbarbu15
* status: in progress (5%)
* owner: aya
* status: in progress (50%)
* start-date: 2024/11/18
* end-date: 2024/12/02

Expand All @@ -155,8 +155,8 @@ without compromising coverage.
### CI Integration

* fully qualified name: `vac:qa:waku:interop-testing-2024q4:ci-integration`
* owner: fbarbu15
* status: not started
* owner: aya
* status: in progress (50%)
* start-date: 2024/12/02
* end-date: 2024/12/09

Expand Down Expand Up @@ -193,3 +193,55 @@ to catch issues that may arise under suboptimal network conditions.
* CI configuration for on-demand full test runs with network constraints
* Nightly CI runs for a subset of tests under network strain

### Maintenance

* fully qualified name: `vac:qa:waku:interop-testing-2024q4:maintenace`
* owner: fbarbu15
* status: started (50%)
* start-date: 2024/10/01
* end-date: 2024/12/31

#### Description
Ongoing Maintenance for Waku Interop Tests
This task involves ensuring the continued reliability
and functionality of the Waku interop tests.
It includes adjusting tests whenever there are changes in nwaku or go-waku,
addressing detected regressions through investigation, reporting, and retesting,
and resolving infrastructure issues that disrupt the tests.
Additionally, it covers unplanned but necessary minor improvements to the testing framework
to enhance its effectiveness.

#### Deliverables

PRs:
* https://github.com/waku-org/waku-interop-tests/pull/75
* https://github.com/waku-org/waku-interop-tests/pull/79
* https://github.com/waku-org/waku-interop-tests/pull/82
* https://github.com/waku-org/waku-interop-tests/pull/84
* https://github.com/waku-org/waku-interop-tests/pull/91

Issues found:
* https://github.com/waku-org/go-waku/issues/1255

### Libwaku Discovery

* fully qualified name: `vac:qa:waku:interop-testing-2024q4:discover-libwaku`
* owner: aya
* status: started (50%)
* start-date: 2024/11/25
* end-date: 2024/11/28

#### Description
Brief Discovery for Libwaku Python and Golang Bindings
Waku developers have requested support in evaluating
the possibility of creating a test framework for libwaku testing.
This task involves dedicating a few days to explore the requirements,
assess the feasibility, and determine whether we have the necessary capabilities
to work effectively with the existing implementation.
The goal is to understand the scope of the task
and identify the next steps for potential framework development.

#### Deliverables
PRs:

Issues found: