[BUG] SMP signal dispatch seems racy

[yamt]

Description / Steps to reproduce the issue

the signal sending logic (kill) basically does:

1. enter_critical_section
2. decide the target thread
3. check signal mask of the thread
4. make a cross call to the target cpu (nxsched_smp_call_single)

as nxsched_smp_call_single drops the critical section, there seems to be nothing to prevent the signal mask from changing in the meantime. it can end up with a signal delivered while being blocked.

similarly, the need_restore logic in sig_handler seems dangerous as it can revert unrelated changes to those fields.

On which OS does this issue occur?

[OS: Mac]

What is the version of your OS?

macOS 14.7

NuttX Version

master

Issue Architecture

[Arch: all]

Issue Area

[Area: Kernel]

Verification

• I have verified before submitting the report.

[xiaoxiang781216]

@hujun260 please look at this issue.

[hujun260]

as nxsched_smp_call_single drops the critical section, there seems to be nothing to prevent the signal mask from changing in the meantime. it can end up with a signal delivered while being blocked.

I don't think this change caused #14749,
because the release of the critical section is going to happen regardless of whether I call nxsched_smp_call_single or not.

similarly, the need_restore logic in sig_handler seems dangerous as it can revert unrelated changes to those fields.

There are two fields: affinity and flag.
Due to the setting of TCB_FLAG_CPU_LOCKED, the affinity will not be changed during the process.

The tcb->flags may undergo changes, which I can further refine. During restoration, only the TCB_FLAG_CPU_LOCKED field of tcb->flags will be restored, without altering any other fields.
#14812

[yamt]

as nxsched_smp_call_single drops the critical section, there seems to be nothing to prevent the signal mask from changing in the meantime. it can end up with a signal delivered while being blocked.

I don't think this change caused #14749, because the release of the critical section is going to happen regardless of whether I call nxsched_smp_call_single or not.

i don't understand your logic.
in non cross call case, the critical section is released after up_schedule_sigaction.

[hujun260]

in non cross call case, the critical section is released after up_schedule_sigaction.

Regardless of whether it crosses cores, up_schedule_sigaction will be invoked, and nxsched_smp_call_single will wait until up_schedule_sigaction has been called and returns.

[yamt]

in non cross call case, the critical section is released after up_schedule_sigaction.

Regardless of whether it crosses cores, up_schedule_sigaction will be invoked, and nxsched_smp_call_single will wait until up_schedule_sigaction has been called and returns.

a cross call releases the critical section before calling up_schedule_sigaction. thus racy.

[hujun260]

a cross call releases the critical section before calling up_schedule_sigaction. thus racy.

The vulnerability appears to be persistent, even if up_schedule_sigaction is called immediately in critical section,
it still releases the critical section immediately and cannot prevent the signal mask from changing